2024-04-26T20:56:06.459 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T20:56:06.459 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675194 --verbose --owner scheduled_rishabh@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/backtrace} --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675194/orig.config.yaml 2024-04-26T20:56:06.504 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T20:56:06.605 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675194 branch: wip-rishabh-testing-20240426.111959 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/backtrace} email: null first_in_suite: false job_id: '7675194' kernel: client: branch: testing flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi026.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN+3caLPLKkWOO8ZXqexY8z9NGh/HH5hyG6O6NGFgQIsSNb1vK3UAgFN+F4JVeuAIvzIIqBo0PW8Hs6qaSmImc8= smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK0wPYqHDPyXwXdZ3vmTmk7/fzwGtg5oeDPlTsCRpzLiKXuAwck96k8Sl4lQrPI6TYMDb1xH6PoiFhNCIAUjjds= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_backtrace teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T20:56:06.605 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T20:56:06.606 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T20:56:06.606 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T20:56:06.607 INFO:teuthology.task.internal:Checking packages... 2024-04-26T20:56:06.630 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T20:56:06.630 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T20:56:06.630 INFO:teuthology.packaging:ref: None 2024-04-26T20:56:06.630 INFO:teuthology.packaging:tag: None 2024-04-26T20:56:06.630 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T20:56:06.630 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:56:06.630 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T20:56:06.771 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T20:56:06.773 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T20:56:06.779 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T20:56:06.779 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T20:56:06.802 INFO:teuthology.task.internal:Saving configuration 2024-04-26T20:56:06.819 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T20:56:06.825 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T20:56:06.851 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675194', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 20:51:04.572469', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN+3caLPLKkWOO8ZXqexY8z9NGh/HH5hyG6O6NGFgQIsSNb1vK3UAgFN+F4JVeuAIvzIIqBo0PW8Hs6qaSmImc8='} 2024-04-26T20:56:06.876 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675194', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 20:51:04.570843', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK0wPYqHDPyXwXdZ3vmTmk7/fzwGtg5oeDPlTsCRpzLiKXuAwck96k8Sl4lQrPI6TYMDb1xH6PoiFhNCIAUjjds='} 2024-04-26T20:56:06.876 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T20:56:06.883 INFO:teuthology.task.internal:roles: ubuntu@smithi026.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-26T20:56:06.883 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-26T20:56:06.883 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T20:56:06.959 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f42163ce160>, signals=[15]) 2024-04-26T20:56:06.960 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T20:56:06.966 INFO:teuthology.task.internal:Opening connections... 2024-04-26T20:56:06.966 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi026.front.sepia.ceph.com 2024-04-26T20:56:06.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:56:07.047 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2024-04-26T20:56:07.048 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:56:07.123 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T20:56:07.129 DEBUG:teuthology.orchestra.run.smithi026:> uname -m 2024-04-26T20:56:07.148 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2024-04-26T20:56:07.148 DEBUG:teuthology.orchestra.run.smithi026:> cat /etc/os-release 2024-04-26T20:56:07.206 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="CentOS Stream" 2024-04-26T20:56:07.206 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="9" 2024-04-26T20:56:07.206 INFO:teuthology.orchestra.run.smithi026.stdout:ID="centos" 2024-04-26T20:56:07.207 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE="rhel fedora" 2024-04-26T20:56:07.207 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="9" 2024-04-26T20:56:07.207 INFO:teuthology.orchestra.run.smithi026.stdout:PLATFORM_ID="platform:el9" 2024-04-26T20:56:07.207 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T20:56:07.207 INFO:teuthology.orchestra.run.smithi026.stdout:ANSI_COLOR="0;31" 2024-04-26T20:56:07.207 INFO:teuthology.orchestra.run.smithi026.stdout:LOGO="fedora-logo-icon" 2024-04-26T20:56:07.207 INFO:teuthology.orchestra.run.smithi026.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T20:56:07.207 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://centos.org/" 2024-04-26T20:56:07.207 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T20:56:07.207 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T20:56:07.207 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T20:56:07.208 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2024-04-26T20:56:07.247 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2024-04-26T20:56:07.264 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2024-04-26T20:56:07.264 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2024-04-26T20:56:07.321 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="CentOS Stream" 2024-04-26T20:56:07.321 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="9" 2024-04-26T20:56:07.322 INFO:teuthology.orchestra.run.smithi086.stdout:ID="centos" 2024-04-26T20:56:07.322 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE="rhel fedora" 2024-04-26T20:56:07.322 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="9" 2024-04-26T20:56:07.322 INFO:teuthology.orchestra.run.smithi086.stdout:PLATFORM_ID="platform:el9" 2024-04-26T20:56:07.322 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T20:56:07.322 INFO:teuthology.orchestra.run.smithi086.stdout:ANSI_COLOR="0;31" 2024-04-26T20:56:07.322 INFO:teuthology.orchestra.run.smithi086.stdout:LOGO="fedora-logo-icon" 2024-04-26T20:56:07.322 INFO:teuthology.orchestra.run.smithi086.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T20:56:07.322 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://centos.org/" 2024-04-26T20:56:07.322 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T20:56:07.322 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T20:56:07.322 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T20:56:07.323 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2024-04-26T20:56:07.344 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T20:56:07.354 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T20:56:07.360 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T20:56:07.360 DEBUG:teuthology.orchestra.run.smithi026:> test '!' -e /home/ubuntu/cephtest 2024-04-26T20:56:07.362 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2024-04-26T20:56:07.378 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T20:56:07.385 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T20:56:07.385 DEBUG:teuthology.orchestra.run.smithi026:> test -z $(ls -A /var/lib/ceph) 2024-04-26T20:56:07.420 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2024-04-26T20:56:07.536 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T20:56:07.595 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T20:56:07.610 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T20:56:07.610 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-04-26T20:56:07.610 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-04-26T20:56:07.611 DEBUG:teuthology.orchestra.run.smithi026:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T20:56:07.611 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T20:56:07.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:56:07.627 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-26T20:56:07.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:56:07.631 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-26T20:56:07.782 DEBUG:teuthology.task.kernel:sha1 for client.1 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:56:07.783 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T20:56:07.783 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-04-26T20:56:07.800 INFO:teuthology.orchestra.run.smithi086.stdout:5.14.0-437.el9.x86_64 2024-04-26T20:56:07.800 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:56:07.800 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T20:56:07.801 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum upgrade -y linux-firmware 2024-04-26T20:56:07.856 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:56:07.856 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T20:56:07.856 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2024-04-26T20:56:07.874 INFO:teuthology.orchestra.run.smithi026.stdout:5.14.0-437.el9.x86_64 2024-04-26T20:56:07.874 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:56:07.874 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T20:56:07.874 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum upgrade -y linux-firmware 2024-04-26T20:56:10.644 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 6.8 MB/s | 8.0 MB 00:01 2024-04-26T20:56:10.798 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - BaseOS 6.3 MB/s | 8.0 MB 00:01 2024-04-26T20:56:12.445 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-04-26T20:56:13.766 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-04-26T20:56:16.804 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 7.9 MB/s | 6.2 MB 00:00 2024-04-26T20:56:18.508 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - CRB 5.9 MB/s | 6.2 MB 00:01 2024-04-26T20:56:18.597 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-04-26T20:56:20.017 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-26T20:56:20.909 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-26T20:56:25.073 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-26T20:56:27.056 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-04-26T20:56:27.057 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-04-26T20:56:27.057 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-04-26T20:56:27.174 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-26T20:56:27.174 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:56:27.399 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T20:56:27.399 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T20:56:27.401 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi086.front.sepia.ceph.com to complete... 2024-04-26T20:56:39.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26 20:56:39 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T20:56:39.560 INFO:teuthology.orchestra.run.smithi086.stderr:FINISHED --2024-04-26 20:56:39-- 2024-04-26T20:56:39.560 INFO:teuthology.orchestra.run.smithi086.stderr:Total wall clock time: 12s 2024-04-26T20:56:39.560 INFO:teuthology.orchestra.run.smithi086.stderr:Downloaded: 1 files, 341M in 12s (28.8 MB/s) 2024-04-26T20:56:39.563 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-26T20:56:39.564 DEBUG:teuthology.orchestra.run.smithi086:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T20:56:40.877 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-04-26T20:56:41.744 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying... ######################################## 2024-04-26T20:56:41.878 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing... ######################################## 2024-04-26T20:56:41.995 INFO:teuthology.orchestra.run.smithi086.stdout:Updating / installing... 2024-04-26T20:56:46.058 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-04-26T20:56:48.103 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-04-26T20:56:48.104 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2024-04-26T20:56:48.104 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-04-26T20:56:48.204 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-26T20:56:48.204 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:56:48.427 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T20:56:48.427 DEBUG:teuthology.orchestra.run.smithi026:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T20:56:48.429 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi026.front.sepia.ceph.com to complete... 2024-04-26T20:56:49.977 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T20:57:24.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26 20:57:24 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T20:57:24.259 INFO:teuthology.orchestra.run.smithi026.stderr:FINISHED --2024-04-26 20:57:24-- 2024-04-26T20:57:24.259 INFO:teuthology.orchestra.run.smithi026.stderr:Total wall clock time: 36s 2024-04-26T20:57:24.259 INFO:teuthology.orchestra.run.smithi026.stderr:Downloaded: 1 files, 341M in 35s (9.68 MB/s) 2024-04-26T20:57:24.262 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-26T20:57:24.263 DEBUG:teuthology.orchestra.run.smithi026:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T20:57:26.450 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying... ######################################## 2024-04-26T20:57:26.587 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing... ######################################## 2024-04-26T20:57:26.688 INFO:teuthology.orchestra.run.smithi026.stdout:Updating / installing... 2024-04-26T20:57:28.788 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi086.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T20:57:28.788 DEBUG:teuthology.orchestra.run.smithi086:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T20:57:28.874 INFO:teuthology.orchestra.run.smithi086.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T20:57:28.874 INFO:teuthology.orchestra.run.smithi086.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T20:57:28.874 INFO:teuthology.orchestra.run.smithi086.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T20:57:28.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T20:57:28.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T20:57:28.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T20:57:28.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T20:57:28.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T20:57:28.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T20:57:28.874 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T20:57:28.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T20:57:28.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T20:57:28.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T20:57:28.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T20:57:28.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T20:57:28.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T20:57:28.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T20:57:28.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T20:57:28.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T20:57:28.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T20:57:28.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T20:57:28.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T20:57:28.875 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T20:57:28.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T20:57:28.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T20:57:28.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T20:57:28.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T20:57:28.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T20:57:28.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T20:57:28.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T20:57:28.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T20:57:28.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T20:57:28.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T20:57:28.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T20:57:28.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T20:57:28.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T20:57:28.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T20:57:28.876 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T20:57:28.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T20:57:28.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T20:57:28.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T20:57:28.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T20:57:28.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T20:57:28.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T20:57:28.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T20:57:28.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T20:57:28.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T20:57:28.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T20:57:28.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T20:57:28.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T20:57:28.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T20:57:28.877 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T20:57:28.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T20:57:28.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T20:57:28.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T20:57:28.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T20:57:28.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T20:57:28.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T20:57:28.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T20:57:28.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T20:57:28.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T20:57:28.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T20:57:28.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T20:57:28.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T20:57:28.878 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T20:57:28.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T20:57:28.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T20:57:28.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T20:57:28.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T20:57:28.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T20:57:28.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T20:57:28.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T20:57:28.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T20:57:28.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T20:57:28.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T20:57:28.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T20:57:28.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T20:57:28.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T20:57:28.879 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T20:57:28.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T20:57:28.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T20:57:28.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T20:57:28.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T20:57:28.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T20:57:28.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T20:57:28.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T20:57:28.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T20:57:28.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T20:57:28.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T20:57:28.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T20:57:28.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T20:57:28.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T20:57:28.880 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T20:57:28.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T20:57:28.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T20:57:28.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T20:57:28.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T20:57:28.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T20:57:28.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T20:57:28.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T20:57:28.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T20:57:28.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T20:57:28.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T20:57:28.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T20:57:28.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T20:57:28.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T20:57:28.881 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T20:57:28.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T20:57:28.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T20:57:28.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T20:57:28.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T20:57:28.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T20:57:28.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T20:57:28.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T20:57:28.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T20:57:28.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T20:57:28.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T20:57:28.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T20:57:28.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T20:57:28.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T20:57:28.882 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T20:57:28.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T20:57:28.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T20:57:28.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T20:57:28.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T20:57:28.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T20:57:28.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T20:57:28.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T20:57:28.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T20:57:28.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T20:57:28.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T20:57:28.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T20:57:28.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T20:57:28.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T20:57:28.883 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T20:57:28.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T20:57:28.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T20:57:28.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T20:57:28.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T20:57:28.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T20:57:28.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T20:57:28.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T20:57:28.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T20:57:28.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T20:57:28.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T20:57:28.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T20:57:28.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T20:57:28.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T20:57:28.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T20:57:28.884 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T20:57:28.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T20:57:28.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T20:57:28.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T20:57:28.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T20:57:28.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T20:57:28.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T20:57:28.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T20:57:28.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T20:57:28.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T20:57:28.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T20:57:28.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T20:57:28.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T20:57:28.885 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T20:57:28.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T20:57:28.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T20:57:28.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T20:57:28.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T20:57:28.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T20:57:28.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T20:57:28.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T20:57:28.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T20:57:28.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T20:57:28.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T20:57:28.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T20:57:28.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T20:57:28.886 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T20:57:28.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T20:57:28.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T20:57:28.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T20:57:28.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T20:57:28.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T20:57:28.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T20:57:28.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T20:57:28.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T20:57:28.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T20:57:28.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T20:57:28.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T20:57:28.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T20:57:28.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T20:57:28.887 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T20:57:28.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T20:57:28.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T20:57:28.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T20:57:28.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T20:57:28.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T20:57:28.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T20:57:28.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T20:57:28.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T20:57:28.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T20:57:28.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T20:57:28.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T20:57:28.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T20:57:28.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T20:57:28.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T20:57:28.888 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T20:57:28.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T20:57:28.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T20:57:28.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T20:57:28.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T20:57:28.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T20:57:28.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T20:57:28.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T20:57:28.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T20:57:28.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T20:57:28.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T20:57:28.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T20:57:28.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T20:57:28.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T20:57:28.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T20:57:28.889 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T20:57:28.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T20:57:28.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T20:57:28.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T20:57:28.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T20:57:28.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T20:57:28.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T20:57:28.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T20:57:28.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T20:57:28.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T20:57:28.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T20:57:28.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T20:57:28.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T20:57:28.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T20:57:28.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T20:57:28.890 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T20:57:28.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T20:57:28.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T20:57:28.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T20:57:28.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T20:57:28.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T20:57:28.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T20:57:28.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T20:57:28.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T20:57:28.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T20:57:28.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T20:57:28.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T20:57:28.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T20:57:28.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T20:57:28.891 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T20:57:28.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T20:57:28.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T20:57:28.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T20:57:28.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T20:57:28.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T20:57:28.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T20:57:28.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T20:57:28.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T20:57:28.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T20:57:28.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T20:57:28.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T20:57:28.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T20:57:28.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T20:57:28.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T20:57:28.892 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T20:57:28.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T20:57:28.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T20:57:28.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T20:57:28.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T20:57:28.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T20:57:28.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T20:57:28.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T20:57:28.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T20:57:28.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T20:57:28.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T20:57:28.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T20:57:28.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T20:57:28.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T20:57:28.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T20:57:28.893 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T20:57:28.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T20:57:28.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T20:57:28.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T20:57:28.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T20:57:28.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T20:57:28.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T20:57:28.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T20:57:28.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T20:57:28.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T20:57:28.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T20:57:28.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T20:57:28.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T20:57:28.894 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T20:57:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T20:57:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T20:57:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T20:57:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T20:57:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T20:57:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T20:57:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T20:57:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T20:57:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T20:57:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T20:57:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T20:57:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T20:57:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T20:57:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T20:57:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T20:57:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T20:57:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T20:57:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T20:57:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T20:57:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T20:57:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T20:57:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T20:57:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T20:57:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T20:57:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T20:57:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T20:57:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T20:57:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T20:57:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T20:57:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T20:57:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T20:57:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T20:57:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T20:57:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T20:57:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T20:57:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T20:57:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T20:57:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T20:57:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T20:57:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T20:57:28.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T20:57:28.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T20:57:28.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T20:57:28.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T20:57:28.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T20:57:28.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T20:57:28.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T20:57:28.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T20:57:28.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T20:57:28.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T20:57:28.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T20:57:28.898 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T20:57:28.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T20:57:28.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T20:57:28.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T20:57:28.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T20:57:28.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T20:57:28.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T20:57:28.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T20:57:28.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T20:57:28.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T20:57:28.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T20:57:28.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T20:57:28.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T20:57:28.899 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T20:57:28.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T20:57:28.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T20:57:28.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T20:57:28.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T20:57:28.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T20:57:28.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T20:57:28.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T20:57:28.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T20:57:28.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T20:57:28.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T20:57:28.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T20:57:28.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T20:57:28.900 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T20:57:28.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T20:57:28.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T20:57:28.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T20:57:28.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T20:57:28.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T20:57:28.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T20:57:28.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T20:57:28.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T20:57:28.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T20:57:28.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T20:57:28.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T20:57:28.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T20:57:28.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T20:57:28.901 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T20:57:28.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T20:57:28.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T20:57:28.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T20:57:28.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T20:57:28.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T20:57:28.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T20:57:28.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T20:57:28.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T20:57:28.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T20:57:28.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T20:57:28.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T20:57:28.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T20:57:28.902 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T20:57:28.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T20:57:28.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T20:57:28.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T20:57:28.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T20:57:28.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T20:57:28.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T20:57:28.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T20:57:28.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T20:57:28.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T20:57:28.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T20:57:28.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T20:57:28.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T20:57:28.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T20:57:28.903 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T20:57:28.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T20:57:28.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T20:57:28.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T20:57:28.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T20:57:28.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T20:57:28.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T20:57:28.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T20:57:28.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T20:57:28.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T20:57:28.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T20:57:28.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T20:57:28.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T20:57:28.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T20:57:28.904 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T20:57:28.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T20:57:28.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T20:57:28.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T20:57:28.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T20:57:28.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T20:57:28.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T20:57:28.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T20:57:28.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T20:57:28.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T20:57:28.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T20:57:28.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T20:57:28.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T20:57:28.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T20:57:28.905 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T20:57:28.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T20:57:28.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T20:57:28.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T20:57:28.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T20:57:28.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T20:57:28.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T20:57:28.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T20:57:28.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T20:57:28.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T20:57:28.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T20:57:28.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T20:57:28.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T20:57:28.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T20:57:28.906 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T20:57:28.907 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T20:57:28.907 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T20:57:28.907 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T20:57:28.907 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T20:57:28.909 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T20:57:28.909 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T20:57:28.909 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T20:57:28.909 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T20:57:28.909 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T20:57:28.909 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T20:57:28.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T20:57:28.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T20:57:28.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T20:57:28.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T20:57:28.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T20:57:28.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T20:57:28.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T20:57:28.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T20:57:28.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T20:57:28.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T20:57:28.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T20:57:28.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T20:57:28.910 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T20:57:28.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T20:57:28.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T20:57:28.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T20:57:28.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T20:57:28.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T20:57:28.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T20:57:28.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T20:57:28.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T20:57:28.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T20:57:28.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T20:57:28.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T20:57:28.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T20:57:28.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T20:57:28.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T20:57:28.911 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T20:57:28.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T20:57:28.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T20:57:28.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T20:57:28.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T20:57:28.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T20:57:28.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T20:57:28.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T20:57:28.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T20:57:28.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T20:57:28.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T20:57:28.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T20:57:28.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T20:57:28.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T20:57:28.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T20:57:28.912 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T20:57:28.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T20:57:28.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T20:57:28.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T20:57:28.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T20:57:28.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T20:57:28.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T20:57:28.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T20:57:28.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T20:57:28.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T20:57:28.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T20:57:28.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T20:57:28.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T20:57:28.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T20:57:28.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T20:57:28.913 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T20:57:28.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T20:57:28.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T20:57:28.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T20:57:28.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T20:57:28.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T20:57:28.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T20:57:28.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T20:57:28.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T20:57:28.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T20:57:28.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T20:57:28.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T20:57:28.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T20:57:28.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T20:57:28.914 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T20:57:28.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T20:57:28.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T20:57:28.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T20:57:28.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T20:57:28.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T20:57:28.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T20:57:28.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T20:57:28.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T20:57:28.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T20:57:28.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T20:57:28.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T20:57:28.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T20:57:28.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T20:57:28.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T20:57:28.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T20:57:28.915 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T20:57:28.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T20:57:28.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T20:57:28.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T20:57:28.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T20:57:28.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T20:57:28.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T20:57:28.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T20:57:28.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T20:57:28.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T20:57:28.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T20:57:28.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T20:57:28.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T20:57:28.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T20:57:28.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T20:57:28.916 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T20:57:28.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T20:57:28.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T20:57:28.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T20:57:28.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T20:57:28.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T20:57:28.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T20:57:28.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T20:57:28.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T20:57:28.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T20:57:28.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T20:57:28.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T20:57:28.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T20:57:28.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T20:57:28.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T20:57:28.917 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T20:57:28.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T20:57:28.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T20:57:28.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T20:57:28.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T20:57:28.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T20:57:28.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T20:57:28.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T20:57:28.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T20:57:28.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T20:57:28.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T20:57:28.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T20:57:28.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T20:57:28.918 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T20:57:28.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T20:57:28.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T20:57:28.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T20:57:28.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T20:57:28.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T20:57:28.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T20:57:28.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T20:57:28.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T20:57:28.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T20:57:28.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T20:57:28.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T20:57:28.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T20:57:28.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T20:57:28.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T20:57:28.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T20:57:28.919 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T20:57:28.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T20:57:28.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T20:57:28.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T20:57:28.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T20:57:28.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T20:57:28.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T20:57:28.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T20:57:28.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T20:57:28.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T20:57:28.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T20:57:28.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T20:57:28.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T20:57:28.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T20:57:28.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T20:57:28.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T20:57:28.920 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T20:57:28.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T20:57:28.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T20:57:28.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T20:57:28.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T20:57:28.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T20:57:28.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T20:57:28.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T20:57:28.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T20:57:28.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T20:57:28.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T20:57:28.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T20:57:28.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T20:57:28.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T20:57:28.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T20:57:28.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T20:57:28.921 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T20:57:28.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T20:57:28.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T20:57:28.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T20:57:28.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T20:57:28.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T20:57:28.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T20:57:28.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T20:57:28.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T20:57:28.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T20:57:28.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T20:57:28.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T20:57:28.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T20:57:28.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T20:57:28.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T20:57:28.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T20:57:28.922 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T20:57:28.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T20:57:28.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T20:57:28.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T20:57:28.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T20:57:28.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T20:57:28.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T20:57:28.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T20:57:28.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T20:57:28.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T20:57:28.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T20:57:28.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T20:57:28.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T20:57:28.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T20:57:28.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T20:57:28.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T20:57:28.923 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T20:57:28.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T20:57:28.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T20:57:28.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T20:57:28.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T20:57:28.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T20:57:28.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T20:57:28.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T20:57:28.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T20:57:28.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T20:57:28.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T20:57:28.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T20:57:28.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T20:57:28.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T20:57:28.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T20:57:28.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T20:57:28.924 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T20:57:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T20:57:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T20:57:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T20:57:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T20:57:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T20:57:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T20:57:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T20:57:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T20:57:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T20:57:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T20:57:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T20:57:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T20:57:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T20:57:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T20:57:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T20:57:28.925 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T20:57:28.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T20:57:28.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T20:57:28.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T20:57:28.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T20:57:28.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T20:57:28.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T20:57:28.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T20:57:28.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T20:57:28.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T20:57:28.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T20:57:28.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T20:57:28.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T20:57:28.926 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T20:57:28.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T20:57:28.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T20:57:28.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T20:57:28.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T20:57:28.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T20:57:28.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T20:57:28.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T20:57:28.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T20:57:28.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T20:57:28.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T20:57:28.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T20:57:28.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T20:57:28.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T20:57:28.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T20:57:28.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T20:57:28.927 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T20:57:28.928 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T20:57:28.928 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T20:57:28.928 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T20:57:28.928 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T20:57:28.928 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T20:57:28.928 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T20:57:28.928 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T20:57:28.928 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T20:57:28.928 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T20:57:28.928 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T20:57:28.928 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T20:57:28.928 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T20:57:28.928 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T20:57:28.928 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T20:57:28.928 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T20:57:28.928 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T20:57:28.929 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T20:57:28.929 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T20:57:28.929 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T20:57:28.929 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T20:57:28.929 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T20:57:28.929 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T20:57:28.929 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T20:57:28.929 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T20:57:28.929 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T20:57:28.929 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T20:57:28.929 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T20:57:28.929 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T20:57:28.929 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T20:57:28.929 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T20:57:28.929 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T20:57:28.929 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T20:57:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T20:57:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T20:57:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T20:57:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T20:57:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T20:57:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T20:57:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T20:57:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T20:57:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T20:57:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T20:57:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T20:57:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T20:57:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T20:57:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T20:57:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T20:57:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T20:57:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T20:57:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T20:57:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T20:57:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T20:57:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T20:57:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T20:57:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T20:57:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T20:57:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T20:57:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T20:57:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T20:57:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T20:57:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T20:57:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T20:57:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T20:57:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T20:57:28.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T20:57:28.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T20:57:28.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T20:57:28.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T20:57:28.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T20:57:28.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T20:57:28.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T20:57:28.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T20:57:28.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T20:57:28.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T20:57:28.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T20:57:28.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T20:57:28.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T20:57:28.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T20:57:28.932 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T20:57:28.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T20:57:28.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T20:57:28.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T20:57:28.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T20:57:28.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T20:57:28.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T20:57:28.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T20:57:28.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T20:57:28.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T20:57:28.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T20:57:28.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T20:57:28.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T20:57:28.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T20:57:28.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T20:57:28.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T20:57:28.933 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T20:57:28.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T20:57:28.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T20:57:28.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T20:57:28.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T20:57:28.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T20:57:28.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T20:57:28.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T20:57:28.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T20:57:28.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T20:57:28.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T20:57:28.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T20:57:28.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T20:57:28.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T20:57:28.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T20:57:28.934 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T20:57:28.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T20:57:28.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T20:57:28.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T20:57:28.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T20:57:28.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T20:57:28.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T20:57:28.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T20:57:28.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T20:57:28.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T20:57:28.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T20:57:28.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T20:57:28.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T20:57:28.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T20:57:28.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T20:57:28.935 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T20:57:28.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T20:57:28.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T20:57:28.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T20:57:28.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T20:57:28.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T20:57:28.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T20:57:28.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T20:57:28.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T20:57:28.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T20:57:28.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T20:57:28.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T20:57:28.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T20:57:28.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T20:57:28.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T20:57:28.936 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T20:57:28.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T20:57:28.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T20:57:28.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T20:57:28.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T20:57:28.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T20:57:28.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T20:57:28.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T20:57:28.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T20:57:28.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T20:57:28.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T20:57:28.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T20:57:28.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T20:57:28.937 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T20:57:28.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T20:57:28.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T20:57:28.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T20:57:28.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T20:57:28.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T20:57:28.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T20:57:28.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T20:57:28.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T20:57:28.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T20:57:28.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T20:57:28.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T20:57:28.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T20:57:28.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T20:57:28.938 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T20:57:28.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T20:57:28.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T20:57:28.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T20:57:28.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T20:57:28.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T20:57:28.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T20:57:28.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T20:57:28.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T20:57:28.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T20:57:28.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T20:57:28.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T20:57:28.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T20:57:28.939 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T20:57:28.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T20:57:28.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T20:57:28.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T20:57:28.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T20:57:28.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T20:57:28.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T20:57:28.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T20:57:28.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T20:57:28.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T20:57:28.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T20:57:28.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T20:57:28.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T20:57:28.940 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T20:57:28.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T20:57:28.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T20:57:28.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T20:57:28.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T20:57:28.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T20:57:28.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T20:57:28.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T20:57:28.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T20:57:28.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T20:57:28.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T20:57:28.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T20:57:28.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T20:57:28.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T20:57:28.941 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T20:57:28.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T20:57:28.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T20:57:28.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T20:57:28.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T20:57:28.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T20:57:28.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T20:57:28.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T20:57:28.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T20:57:28.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T20:57:28.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T20:57:28.942 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T20:57:28.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T20:57:28.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T20:57:28.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T20:57:28.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T20:57:28.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T20:57:28.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T20:57:28.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T20:57:28.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T20:57:28.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T20:57:28.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T20:57:28.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T20:57:28.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T20:57:28.943 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T20:57:28.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T20:57:28.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T20:57:28.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T20:57:28.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T20:57:28.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T20:57:28.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T20:57:28.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T20:57:28.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T20:57:28.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T20:57:28.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T20:57:28.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T20:57:28.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T20:57:28.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T20:57:28.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T20:57:28.944 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T20:57:28.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T20:57:28.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T20:57:28.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T20:57:28.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T20:57:28.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T20:57:28.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T20:57:28.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T20:57:28.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T20:57:28.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T20:57:28.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T20:57:28.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T20:57:28.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T20:57:28.945 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T20:57:28.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T20:57:28.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T20:57:28.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T20:57:28.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T20:57:28.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T20:57:28.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T20:57:28.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T20:57:28.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T20:57:28.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T20:57:28.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T20:57:28.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T20:57:28.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T20:57:28.946 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T20:57:28.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T20:57:28.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T20:57:28.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T20:57:28.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T20:57:28.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T20:57:28.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T20:57:28.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T20:57:28.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T20:57:28.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T20:57:28.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T20:57:28.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T20:57:28.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T20:57:28.947 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T20:57:28.949 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T20:57:28.949 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T20:57:28.949 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T20:57:28.949 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T20:57:28.949 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T20:57:28.949 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T20:57:28.949 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T20:57:28.949 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T20:57:28.949 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T20:57:28.949 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T20:57:28.949 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T20:57:28.949 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T20:57:28.949 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T20:57:28.949 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T20:57:28.950 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T20:57:28.950 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T20:57:28.950 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T20:57:28.950 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T20:57:28.950 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T20:57:28.950 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T20:57:28.950 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T20:57:28.950 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T20:57:28.950 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T20:57:28.950 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T20:57:28.950 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T20:57:28.950 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T20:57:28.950 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T20:57:28.950 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T20:57:28.951 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T20:57:28.951 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T20:57:28.951 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T20:57:28.951 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T20:57:28.951 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T20:57:28.951 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T20:57:28.951 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T20:57:28.951 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T20:57:28.951 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T20:57:28.951 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T20:57:28.951 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T20:57:28.951 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T20:57:28.951 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T20:57:28.952 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T20:57:28.952 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T20:57:28.952 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T20:57:28.952 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T20:57:28.952 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T20:57:28.952 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T20:57:28.952 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T20:57:28.952 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T20:57:28.952 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T20:57:28.952 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T20:57:28.952 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T20:57:28.952 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T20:57:28.952 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T20:57:28.952 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T20:57:28.953 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T20:57:28.953 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T20:57:28.953 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T20:57:28.953 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T20:57:28.953 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T20:57:28.953 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T20:57:28.953 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T20:57:28.953 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T20:57:28.953 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T20:57:28.953 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T20:57:28.953 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T20:57:28.953 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T20:57:28.953 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T20:57:28.953 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T20:57:28.953 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T20:57:28.954 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T20:57:28.954 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T20:57:28.954 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T20:57:28.954 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T20:57:28.954 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T20:57:28.954 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T20:57:28.954 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T20:57:28.954 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T20:57:28.954 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T20:57:28.954 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T20:57:28.954 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T20:57:28.954 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T20:57:28.954 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T20:57:28.954 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T20:57:28.954 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T20:57:28.955 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T20:57:28.955 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T20:57:28.955 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T20:57:28.955 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T20:57:28.955 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T20:57:28.955 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T20:57:28.955 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T20:57:28.955 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T20:57:28.955 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T20:57:28.955 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T20:57:28.955 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T20:57:28.955 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T20:57:28.955 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T20:57:28.955 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T20:57:28.955 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T20:57:28.955 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T20:57:28.956 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T20:57:28.956 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T20:57:28.956 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T20:57:28.956 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T20:57:28.956 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T20:57:28.956 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T20:57:28.956 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T20:57:28.956 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T20:57:28.956 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T20:57:28.956 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T20:57:28.956 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T20:57:28.956 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T20:57:28.956 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T20:57:28.956 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T20:57:28.957 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T20:57:28.957 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T20:57:28.957 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T20:57:28.957 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T20:57:28.957 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T20:57:28.957 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T20:57:28.957 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T20:57:28.957 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T20:57:28.957 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T20:57:28.957 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T20:57:28.957 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T20:57:28.957 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T20:57:28.957 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T20:57:28.957 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T20:57:28.957 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T20:57:28.958 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T20:57:28.958 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T20:57:28.958 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T20:57:28.958 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T20:57:28.958 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T20:57:28.958 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T20:57:28.958 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T20:57:28.958 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T20:57:28.958 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T20:57:28.958 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T20:57:28.958 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T20:57:28.958 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T20:57:28.958 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T20:57:28.958 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T20:57:28.958 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T20:57:28.959 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T20:57:28.959 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T20:57:28.959 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T20:57:28.959 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T20:57:28.959 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T20:57:28.959 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T20:57:28.959 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T20:57:28.959 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T20:57:28.959 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T20:57:28.959 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T20:57:28.959 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T20:57:28.959 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T20:57:28.959 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T20:57:28.960 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T20:57:28.960 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T20:57:28.960 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T20:57:28.960 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T20:57:28.960 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T20:57:28.960 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T20:57:28.960 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T20:57:28.960 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T20:57:28.960 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T20:57:28.960 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T20:57:28.960 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T20:57:28.960 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T20:57:28.960 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T20:57:28.960 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T20:57:28.960 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T20:57:28.960 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T20:57:28.961 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T20:57:28.961 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T20:57:28.961 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T20:57:28.961 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T20:57:28.961 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T20:57:28.961 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T20:57:28.961 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T20:57:28.961 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T20:57:28.961 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T20:57:28.961 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T20:57:28.961 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T20:57:28.961 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T20:57:28.962 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T20:57:28.962 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T20:57:28.962 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T20:57:28.962 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T20:57:28.962 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T20:57:28.962 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T20:57:28.962 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T20:57:28.962 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T20:57:28.962 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T20:57:28.962 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T20:57:28.962 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T20:57:28.962 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T20:57:28.962 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T20:57:28.962 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T20:57:28.963 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T20:57:28.963 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T20:57:28.963 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T20:57:28.963 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T20:57:28.963 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T20:57:28.963 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T20:57:28.963 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T20:57:28.963 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T20:57:28.963 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T20:57:28.963 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T20:57:28.963 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T20:57:28.963 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T20:57:28.963 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T20:57:28.963 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T20:57:28.964 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T20:57:28.964 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T20:57:28.964 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T20:57:28.964 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T20:57:28.964 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T20:57:28.964 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T20:57:28.964 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T20:57:28.964 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T20:57:28.964 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T20:57:28.964 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T20:57:28.964 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T20:57:28.964 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T20:57:28.965 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T20:57:28.965 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T20:57:28.965 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T20:57:28.965 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T20:57:28.965 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T20:57:28.965 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T20:57:28.965 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T20:57:28.965 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T20:57:28.965 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T20:57:28.965 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T20:57:28.965 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T20:57:28.965 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T20:57:28.965 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T20:57:28.966 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T20:57:28.966 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T20:57:28.966 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T20:57:28.966 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T20:57:28.966 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T20:57:28.966 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T20:57:28.966 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T20:57:28.966 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T20:57:28.966 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T20:57:28.966 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T20:57:28.966 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T20:57:28.966 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T20:57:28.967 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T20:57:28.967 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T20:57:28.967 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T20:57:28.967 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T20:57:28.967 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T20:57:28.967 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T20:57:28.967 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T20:57:28.967 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T20:57:28.967 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T20:57:28.967 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T20:57:28.967 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T20:57:28.967 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T20:57:28.967 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T20:57:28.967 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T20:57:28.967 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T20:57:28.968 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T20:57:28.968 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T20:57:28.968 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T20:57:28.968 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T20:57:28.968 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T20:57:28.968 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T20:57:28.968 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T20:57:28.968 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T20:57:28.968 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T20:57:28.968 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T20:57:28.968 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T20:57:28.968 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T20:57:28.968 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T20:57:28.968 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T20:57:28.968 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T20:57:28.968 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T20:57:28.969 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T20:57:28.969 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T20:57:28.969 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T20:57:28.969 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T20:57:28.969 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T20:57:28.969 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T20:57:28.969 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T20:57:28.969 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T20:57:28.969 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T20:57:28.969 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T20:57:28.969 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T20:57:28.969 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T20:57:28.969 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T20:57:28.970 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T20:57:28.970 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T20:57:28.970 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T20:57:28.970 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T20:57:28.970 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T20:57:28.970 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T20:57:28.970 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T20:57:28.970 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T20:57:28.970 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T20:57:28.970 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T20:57:28.970 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T20:57:28.970 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T20:57:28.970 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T20:57:28.970 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T20:57:28.971 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T20:57:28.971 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T20:57:28.971 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T20:57:28.971 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T20:57:28.971 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T20:57:28.971 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T20:57:28.971 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T20:57:28.971 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T20:57:28.971 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T20:57:28.971 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T20:57:28.971 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T20:57:28.971 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T20:57:28.971 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T20:57:28.971 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T20:57:28.971 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T20:57:28.971 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T20:57:28.972 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T20:57:28.972 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T20:57:28.972 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T20:57:28.972 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T20:57:28.972 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T20:57:28.972 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T20:57:28.972 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T20:57:28.972 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T20:57:28.972 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T20:57:28.972 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T20:57:28.972 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T20:57:28.972 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T20:57:28.973 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T20:57:28.973 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T20:57:28.973 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T20:57:28.973 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T20:57:28.973 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T20:57:28.973 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T20:57:28.973 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T20:57:28.973 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T20:57:28.973 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T20:57:28.973 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T20:57:28.973 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T20:57:28.974 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T20:57:28.974 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T20:57:28.974 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T20:57:28.974 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T20:57:28.974 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T20:57:28.974 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T20:57:28.974 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T20:57:28.974 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T20:57:28.974 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T20:57:28.974 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T20:57:28.974 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T20:57:28.974 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T20:57:28.974 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T20:57:28.974 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T20:57:28.975 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T20:57:28.975 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T20:57:28.975 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T20:57:28.975 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T20:57:28.975 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T20:57:28.975 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T20:57:28.975 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T20:57:28.975 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T20:57:28.975 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T20:57:28.975 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T20:57:28.975 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T20:57:28.975 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T20:57:28.975 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T20:57:28.975 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T20:57:28.976 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T20:57:28.976 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T20:57:28.976 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T20:57:28.976 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T20:57:28.976 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T20:57:28.976 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T20:57:28.976 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T20:57:28.976 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T20:57:28.976 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T20:57:28.976 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T20:57:28.976 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T20:57:28.976 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T20:57:28.976 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T20:57:28.976 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T20:57:28.976 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T20:57:28.977 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T20:57:28.977 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T20:57:28.977 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T20:57:28.977 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T20:57:28.977 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T20:57:28.977 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T20:57:28.977 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T20:57:28.977 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T20:57:28.977 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T20:57:28.977 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T20:57:28.977 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T20:57:28.977 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T20:57:28.977 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T20:57:28.977 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T20:57:28.978 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T20:57:28.978 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T20:57:28.978 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T20:57:28.978 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T20:57:28.978 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T20:57:28.978 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T20:57:28.978 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T20:57:28.978 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T20:57:28.978 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T20:57:28.978 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T20:57:28.978 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T20:57:28.978 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T20:57:28.978 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T20:57:28.978 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T20:57:28.979 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T20:57:28.979 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T20:57:28.979 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T20:57:28.979 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T20:57:28.979 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T20:57:28.979 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T20:57:28.979 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T20:57:28.979 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T20:57:28.979 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T20:57:28.979 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T20:57:28.979 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T20:57:28.979 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T20:57:28.979 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T20:57:28.980 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T20:57:28.980 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T20:57:28.980 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T20:57:28.980 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T20:57:28.980 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T20:57:28.980 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T20:57:28.980 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T20:57:28.980 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T20:57:28.980 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T20:57:28.980 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T20:57:28.980 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T20:57:28.980 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T20:57:28.980 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T20:57:28.980 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T20:57:28.980 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T20:57:28.981 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T20:57:28.981 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T20:57:28.981 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T20:57:28.981 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T20:57:28.981 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T20:57:28.981 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T20:57:28.981 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T20:57:28.981 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T20:57:28.981 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T20:57:28.981 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T20:57:28.981 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T20:57:28.981 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T20:57:28.981 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T20:57:28.982 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T20:57:28.982 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T20:57:28.982 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T20:57:28.982 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T20:57:28.982 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T20:57:28.982 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T20:57:28.982 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T20:57:28.982 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T20:57:28.982 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T20:57:28.982 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T20:57:28.982 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T20:57:28.982 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T20:57:28.983 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T20:57:28.983 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T20:57:28.983 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T20:57:28.983 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T20:57:28.983 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T20:57:28.983 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T20:57:28.983 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T20:57:28.983 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T20:57:28.983 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T20:57:28.983 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T20:57:28.983 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T20:57:28.983 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T20:57:28.983 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T20:57:28.983 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T20:57:28.984 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T20:57:28.984 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T20:57:28.984 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T20:57:28.984 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T20:57:28.984 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T20:57:28.984 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T20:57:28.984 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T20:57:28.984 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T20:57:28.984 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T20:57:28.984 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T20:57:28.984 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T20:57:28.984 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T20:57:28.985 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T20:57:28.985 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T20:57:28.985 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T20:57:28.985 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T20:57:28.985 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T20:57:28.985 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T20:57:28.985 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T20:57:28.985 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T20:57:28.985 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T20:57:28.985 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T20:57:28.985 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T20:57:28.985 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T20:57:28.985 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T20:57:28.985 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T20:57:28.985 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T20:57:28.986 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T20:57:28.986 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T20:57:28.986 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T20:57:28.986 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T20:57:28.986 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T20:57:28.986 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T20:57:28.986 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T20:57:28.986 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T20:57:28.986 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T20:57:28.986 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T20:57:28.986 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T20:57:28.986 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T20:57:28.987 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T20:57:28.987 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T20:57:28.987 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T20:57:28.987 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T20:57:28.987 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T20:57:28.987 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T20:57:28.987 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T20:57:28.987 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T20:57:28.987 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T20:57:28.987 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T20:57:28.987 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T20:57:28.987 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T20:57:28.987 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T20:57:28.987 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T20:57:28.988 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T20:57:28.988 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T20:57:28.988 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T20:57:28.988 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T20:57:28.988 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T20:57:28.988 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T20:57:28.988 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T20:57:28.988 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T20:57:28.988 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T20:57:28.988 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T20:57:28.988 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T20:57:28.988 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T20:57:28.988 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T20:57:28.988 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T20:57:28.988 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T20:57:28.989 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T20:57:28.989 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T20:57:28.989 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T20:57:28.989 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T20:57:28.989 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T20:57:28.989 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T20:57:28.989 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T20:57:28.989 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T20:57:28.989 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T20:57:28.989 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T20:57:28.989 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T20:57:28.989 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T20:57:28.989 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T20:57:28.989 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T20:57:28.989 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T20:57:28.990 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T20:57:28.990 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T20:57:28.990 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T20:57:28.990 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T20:57:28.990 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T20:57:28.990 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T20:57:28.990 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T20:57:28.990 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T20:57:28.990 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T20:57:28.990 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T20:57:28.990 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T20:57:28.990 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T20:57:28.990 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T20:57:28.990 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T20:57:28.991 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T20:57:28.991 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T20:57:28.991 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T20:57:28.991 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T20:57:28.991 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T20:57:28.991 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T20:57:28.991 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T20:57:28.991 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T20:57:28.991 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T20:57:28.991 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T20:57:28.991 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T20:57:28.991 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T20:57:28.991 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T20:57:28.991 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T20:57:28.991 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T20:57:28.991 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T20:57:28.992 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T20:57:28.992 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T20:57:28.992 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T20:57:28.992 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T20:57:28.992 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T20:57:28.992 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T20:57:28.992 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T20:57:28.992 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T20:57:28.992 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T20:57:28.992 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T20:57:28.992 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T20:57:28.992 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T20:57:28.992 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T20:57:28.992 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T20:57:28.992 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T20:57:28.992 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T20:57:28.993 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T20:57:28.993 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T20:57:28.993 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T20:57:28.993 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T20:57:28.993 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T20:57:28.993 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T20:57:28.993 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T20:57:28.993 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T20:57:28.993 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T20:57:28.993 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T20:57:28.993 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T20:57:28.993 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T20:57:28.993 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T20:57:28.993 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T20:57:28.993 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T20:57:28.993 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T20:57:28.993 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T20:57:28.994 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T20:57:28.994 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T20:57:28.994 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T20:57:28.994 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T20:57:28.994 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T20:57:28.994 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T20:57:28.994 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T20:57:28.994 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T20:57:28.994 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T20:57:28.994 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T20:57:28.994 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T20:57:28.994 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T20:57:28.994 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T20:57:28.994 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T20:57:28.994 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T20:57:28.995 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T20:57:28.995 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T20:57:28.995 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T20:57:28.995 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T20:57:28.995 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T20:57:28.995 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T20:57:28.995 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T20:57:28.995 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T20:57:28.995 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T20:57:28.995 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T20:57:28.995 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T20:57:28.995 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T20:57:28.995 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T20:57:28.995 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T20:57:28.995 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T20:57:28.995 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T20:57:28.996 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T20:57:28.996 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T20:57:28.996 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T20:57:28.996 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T20:57:28.996 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T20:57:28.996 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T20:57:28.996 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T20:57:28.996 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T20:57:28.996 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T20:57:28.996 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T20:57:28.996 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T20:57:28.996 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T20:57:28.996 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T20:57:28.996 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T20:57:28.996 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T20:57:28.997 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T20:57:28.997 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T20:57:28.997 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T20:57:28.997 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T20:57:28.997 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T20:57:28.997 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T20:57:28.997 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T20:57:28.997 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T20:57:28.997 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T20:57:28.997 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T20:57:28.997 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T20:57:28.997 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T20:57:28.997 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T20:57:28.997 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T20:57:28.997 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T20:57:28.998 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T20:57:28.998 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T20:57:28.998 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T20:57:28.998 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T20:57:28.998 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T20:57:28.998 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T20:57:28.998 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T20:57:28.998 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T20:57:28.998 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T20:57:28.998 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T20:57:28.998 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T20:57:28.998 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T20:57:28.998 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T20:57:28.998 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T20:57:28.998 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T20:57:28.999 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T20:57:28.999 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T20:57:28.999 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T20:57:28.999 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T20:57:28.999 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T20:57:28.999 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T20:57:28.999 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T20:57:28.999 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T20:57:28.999 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T20:57:28.999 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T20:57:28.999 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T20:57:28.999 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T20:57:28.999 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T20:57:28.999 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T20:57:28.999 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T20:57:28.999 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T20:57:29.000 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T20:57:29.000 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T20:57:29.000 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T20:57:29.000 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T20:57:29.000 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T20:57:29.000 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T20:57:29.000 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T20:57:29.000 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T20:57:29.000 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T20:57:29.000 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T20:57:29.000 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T20:57:29.000 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T20:57:29.000 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T20:57:29.000 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T20:57:29.000 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T20:57:29.001 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T20:57:29.001 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T20:57:29.001 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T20:57:29.001 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T20:57:29.001 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T20:57:29.001 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T20:57:29.001 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T20:57:29.001 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T20:57:29.001 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T20:57:29.001 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T20:57:29.001 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T20:57:29.001 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T20:57:29.001 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T20:57:29.001 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T20:57:29.002 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T20:57:29.002 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T20:57:29.002 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T20:57:29.002 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T20:57:29.002 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T20:57:29.002 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T20:57:29.002 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T20:57:29.002 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T20:57:29.002 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T20:57:29.002 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T20:57:29.002 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T20:57:29.002 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T20:57:29.002 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T20:57:29.002 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T20:57:29.002 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T20:57:29.003 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T20:57:29.003 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T20:57:29.003 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T20:57:29.003 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T20:57:29.003 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T20:57:29.003 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T20:57:29.003 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T20:57:29.003 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T20:57:29.003 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T20:57:29.003 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T20:57:29.003 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T20:57:29.003 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T20:57:29.003 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T20:57:29.003 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T20:57:29.003 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T20:57:29.004 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T20:57:29.004 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T20:57:29.004 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T20:57:29.004 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T20:57:29.004 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T20:57:29.004 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T20:57:29.004 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T20:57:29.004 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T20:57:29.004 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T20:57:29.004 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T20:57:29.004 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T20:57:29.004 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T20:57:29.004 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T20:57:29.004 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T20:57:29.004 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T20:57:29.005 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T20:57:29.005 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T20:57:29.005 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T20:57:29.005 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T20:57:29.005 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T20:57:29.005 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T20:57:29.005 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T20:57:29.005 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T20:57:29.005 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T20:57:29.005 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T20:57:29.005 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T20:57:29.005 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T20:57:29.005 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T20:57:29.005 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T20:57:29.006 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T20:57:29.006 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T20:57:29.006 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T20:57:29.006 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T20:57:29.006 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T20:57:29.006 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T20:57:29.006 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T20:57:29.006 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T20:57:29.006 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T20:57:29.006 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T20:57:29.006 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T20:57:29.006 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T20:57:29.006 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T20:57:29.006 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T20:57:29.007 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T20:57:29.007 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T20:57:29.007 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T20:57:29.007 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T20:57:29.007 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T20:57:29.007 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T20:57:29.007 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T20:57:29.007 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T20:57:29.007 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T20:57:29.007 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T20:57:29.007 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T20:57:29.007 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T20:57:29.007 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T20:57:29.007 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T20:57:29.007 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T20:57:29.008 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T20:57:29.008 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T20:57:29.008 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T20:57:29.008 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T20:57:29.008 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T20:57:29.008 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T20:57:29.008 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T20:57:29.008 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T20:57:29.008 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T20:57:29.008 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T20:57:29.008 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T20:57:29.008 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T20:57:29.008 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T20:57:29.008 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T20:57:29.008 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T20:57:29.008 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T20:57:29.008 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T20:57:29.009 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T20:57:29.009 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T20:57:29.009 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T20:57:29.009 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T20:57:29.009 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T20:57:29.009 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T20:57:29.009 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T20:57:29.009 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T20:57:29.009 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T20:57:29.009 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T20:57:29.009 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T20:57:29.009 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T20:57:29.009 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T20:57:29.009 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T20:57:29.009 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T20:57:29.009 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T20:57:29.010 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T20:57:29.010 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T20:57:29.010 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T20:57:29.010 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T20:57:29.010 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T20:57:29.010 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T20:57:29.010 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T20:57:29.010 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T20:57:29.010 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T20:57:29.010 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T20:57:29.010 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T20:57:29.010 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T20:57:29.010 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T20:57:29.010 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T20:57:29.010 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T20:57:29.011 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T20:57:29.011 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T20:57:29.011 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T20:57:29.011 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T20:57:29.011 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T20:57:29.011 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T20:57:29.011 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T20:57:29.011 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T20:57:29.011 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T20:57:29.011 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T20:57:29.011 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T20:57:29.011 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T20:57:29.011 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T20:57:29.011 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T20:57:29.011 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T20:57:29.011 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T20:57:29.011 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T20:57:29.012 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T20:57:29.012 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T20:57:29.012 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T20:57:29.012 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T20:57:29.012 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T20:57:29.012 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T20:57:29.012 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T20:57:29.016 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T20:57:29.016 DEBUG:teuthology.orchestra.run.smithi086:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T20:57:29.038 INFO:teuthology.orchestra.run.smithi086.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T20:57:29.039 INFO:teuthology.orchestra.run.smithi086.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T20:57:29.039 INFO:teuthology.orchestra.run.smithi086.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T20:57:29.039 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-04-26T20:57:29.039 INFO:teuthology.orchestra.run.smithi086.stdout:for file in vmlinuz System.map config; do 2024-04-26T20:57:29.039 INFO:teuthology.orchestra.run.smithi086.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T20:57:29.039 INFO:teuthology.orchestra.run.smithi086.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:57:29.039 INFO:teuthology.orchestra.run.smithi086.stdout: fi 2024-04-26T20:57:29.039 INFO:teuthology.orchestra.run.smithi086.stdout:done 2024-04-26T20:57:29.039 INFO:teuthology.orchestra.run.smithi086.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T20:57:29.039 INFO:teuthology.orchestra.run.smithi086.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T20:57:29.039 INFO:teuthology.orchestra.run.smithi086.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:57:29.039 INFO:teuthology.orchestra.run.smithi086.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:57:29.039 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-04-26T20:57:29.039 INFO:teuthology.orchestra.run.smithi086.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T20:57:29.040 INFO:teuthology.orchestra.run.smithi086.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T20:57:29.040 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-04-26T20:57:29.040 INFO:teuthology.orchestra.run.smithi086.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T20:57:29.040 INFO:teuthology.orchestra.run.smithi086.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T20:57:29.040 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T20:57:29.040 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-04-26T20:57:29.041 DEBUG:teuthology.orchestra.run.smithi086:> sudo rpm -qi grub2-tools 2024-04-26T20:57:29.122 INFO:teuthology.orchestra.run.smithi086.stdout:Name : grub2-tools 2024-04-26T20:57:29.122 INFO:teuthology.orchestra.run.smithi086.stdout:Epoch : 1 2024-04-26T20:57:29.122 INFO:teuthology.orchestra.run.smithi086.stdout:Version : 2.06 2024-04-26T20:57:29.123 INFO:teuthology.orchestra.run.smithi086.stdout:Release : 68.el9 2024-04-26T20:57:29.123 INFO:teuthology.orchestra.run.smithi086.stdout:Architecture: x86_64 2024-04-26T20:57:29.123 INFO:teuthology.orchestra.run.smithi086.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T20:57:29.123 INFO:teuthology.orchestra.run.smithi086.stdout:Group : Unspecified 2024-04-26T20:57:29.123 INFO:teuthology.orchestra.run.smithi086.stdout:Size : 8264254 2024-04-26T20:57:29.123 INFO:teuthology.orchestra.run.smithi086.stdout:License : GPLv3+ 2024-04-26T20:57:29.123 INFO:teuthology.orchestra.run.smithi086.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T20:57:29.123 INFO:teuthology.orchestra.run.smithi086.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T20:57:29.123 INFO:teuthology.orchestra.run.smithi086.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T20:57:29.123 INFO:teuthology.orchestra.run.smithi086.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T20:57:29.123 INFO:teuthology.orchestra.run.smithi086.stdout:Packager : builder@centos.org 2024-04-26T20:57:29.123 INFO:teuthology.orchestra.run.smithi086.stdout:Vendor : CentOS 2024-04-26T20:57:29.123 INFO:teuthology.orchestra.run.smithi086.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T20:57:29.123 INFO:teuthology.orchestra.run.smithi086.stdout:Summary : Support tools for GRUB. 2024-04-26T20:57:29.123 INFO:teuthology.orchestra.run.smithi086.stdout:Description : 2024-04-26T20:57:29.124 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T20:57:29.124 INFO:teuthology.orchestra.run.smithi086.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T20:57:29.124 INFO:teuthology.orchestra.run.smithi086.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T20:57:29.124 INFO:teuthology.orchestra.run.smithi086.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T20:57:29.124 INFO:teuthology.orchestra.run.smithi086.stdout:hardware devices. 2024-04-26T20:57:29.124 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T20:57:29.124 INFO:teuthology.orchestra.run.smithi086.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T20:57:29.125 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T20:57:29.125 INFO:teuthology.task.kernel:Updating grub on smithi086 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T20:57:29.125 DEBUG:teuthology.orchestra.run.smithi086:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T20:57:29.644 INFO:teuthology.orchestra.run.smithi086.stderr:Generating grub configuration file ... 2024-04-26T20:57:31.052 INFO:teuthology.orchestra.run.smithi086.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T20:57:31.091 INFO:teuthology.orchestra.run.smithi086.stderr:done 2024-04-26T20:57:31.093 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2024-04-26T20:57:31.113 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.NmZV0XvMc4 2024-04-26T20:57:31.113 DEBUG:teuthology.orchestra.run.smithi086:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.NmZV0XvMc4 2024-04-26T20:57:31.195 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.NmZV0XvMc4 2024-04-26T20:57:31.332 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.NmZV0XvMc4 is 6KB 2024-04-26T20:57:31.352 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.NmZV0XvMc4 2024-04-26T20:57:31.366 DEBUG:teuthology.orchestra.run.smithi086:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T20:57:31.427 INFO:teuthology.orchestra.run.smithi086.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T20:57:31.427 INFO:teuthology.orchestra.run.smithi086.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T20:57:31.427 INFO:teuthology.orchestra.run.smithi086.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T20:57:31.427 INFO:teuthology.orchestra.run.smithi086.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T20:57:31.429 DEBUG:teuthology.orchestra.run.smithi086:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T20:57:31.587 DEBUG:teuthology.orchestra.run.smithi086:> sudo shutdown -r now 2024-04-26T20:57:35.004 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T20:58:01.591 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-26T20:58:01.591 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-26T20:58:01.592 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:58:12.908 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi026.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T20:58:12.909 DEBUG:teuthology.orchestra.run.smithi026:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T20:58:12.982 INFO:teuthology.orchestra.run.smithi026.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T20:58:12.982 INFO:teuthology.orchestra.run.smithi026.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T20:58:12.982 INFO:teuthology.orchestra.run.smithi026.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T20:58:12.983 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T20:58:12.983 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T20:58:12.983 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T20:58:12.983 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T20:58:12.983 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T20:58:12.983 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T20:58:12.983 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T20:58:12.983 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T20:58:12.983 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T20:58:12.983 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T20:58:12.983 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T20:58:12.983 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T20:58:12.983 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T20:58:12.983 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T20:58:12.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T20:58:12.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T20:58:12.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T20:58:12.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T20:58:12.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T20:58:12.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T20:58:12.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T20:58:12.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T20:58:12.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T20:58:12.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T20:58:12.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T20:58:12.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T20:58:12.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T20:58:12.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T20:58:12.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T20:58:12.985 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T20:58:12.985 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T20:58:12.985 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T20:58:12.985 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T20:58:12.985 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T20:58:12.985 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T20:58:12.985 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T20:58:12.985 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T20:58:12.985 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T20:58:12.985 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T20:58:12.985 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T20:58:12.985 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T20:58:12.985 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T20:58:12.986 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T20:58:12.986 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T20:58:12.986 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T20:58:12.986 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T20:58:12.986 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T20:58:12.986 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T20:58:12.986 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T20:58:12.986 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T20:58:12.986 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T20:58:12.986 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T20:58:12.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T20:58:12.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T20:58:12.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T20:58:12.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T20:58:12.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T20:58:12.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T20:58:12.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T20:58:12.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T20:58:12.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T20:58:12.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T20:58:12.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T20:58:12.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T20:58:12.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T20:58:12.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T20:58:12.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T20:58:12.988 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T20:58:12.988 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T20:58:12.988 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T20:58:12.988 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T20:58:12.988 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T20:58:12.988 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T20:58:12.988 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T20:58:12.988 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T20:58:12.988 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T20:58:12.988 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T20:58:12.988 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T20:58:12.988 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T20:58:12.988 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T20:58:12.988 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T20:58:12.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T20:58:12.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T20:58:12.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T20:58:12.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T20:58:12.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T20:58:12.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T20:58:12.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T20:58:12.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T20:58:12.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T20:58:12.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T20:58:12.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T20:58:12.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T20:58:12.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T20:58:12.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T20:58:12.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T20:58:12.990 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T20:58:12.990 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T20:58:12.990 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T20:58:12.990 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T20:58:12.990 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T20:58:12.990 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T20:58:12.990 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T20:58:12.990 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T20:58:12.990 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T20:58:12.990 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T20:58:12.990 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T20:58:12.990 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T20:58:12.990 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T20:58:12.991 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T20:58:12.991 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T20:58:12.991 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T20:58:12.991 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T20:58:12.991 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T20:58:12.991 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T20:58:12.991 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T20:58:12.991 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T20:58:12.991 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T20:58:12.991 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T20:58:12.991 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T20:58:12.991 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T20:58:12.991 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T20:58:12.991 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T20:58:12.992 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T20:58:12.992 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T20:58:12.992 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T20:58:12.992 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T20:58:12.992 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T20:58:12.992 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T20:58:12.992 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T20:58:12.992 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T20:58:12.992 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T20:58:12.992 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T20:58:12.992 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T20:58:12.992 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T20:58:12.992 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T20:58:12.993 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T20:58:12.993 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T20:58:12.993 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T20:58:12.993 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T20:58:12.993 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T20:58:12.993 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T20:58:12.993 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T20:58:12.993 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T20:58:12.993 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T20:58:12.993 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T20:58:12.993 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T20:58:12.993 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T20:58:12.993 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T20:58:12.993 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T20:58:12.994 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T20:58:12.994 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T20:58:12.994 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T20:58:12.994 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T20:58:12.994 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T20:58:12.994 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T20:58:12.994 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T20:58:12.994 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T20:58:12.994 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T20:58:12.994 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T20:58:12.994 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T20:58:12.994 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T20:58:12.995 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T20:58:12.995 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T20:58:12.995 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T20:58:12.995 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T20:58:12.995 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T20:58:12.995 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T20:58:12.995 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T20:58:12.995 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T20:58:12.995 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T20:58:12.995 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T20:58:12.995 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T20:58:12.995 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T20:58:12.995 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T20:58:12.995 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T20:58:12.996 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T20:58:12.996 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T20:58:12.996 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T20:58:12.996 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T20:58:12.996 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T20:58:12.996 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T20:58:12.996 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T20:58:12.996 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T20:58:12.996 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T20:58:12.996 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T20:58:12.996 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T20:58:12.996 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T20:58:12.996 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T20:58:12.996 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T20:58:12.996 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T20:58:12.997 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T20:58:12.997 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T20:58:12.997 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T20:58:12.997 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T20:58:12.997 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T20:58:12.997 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T20:58:12.997 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T20:58:12.997 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T20:58:12.997 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T20:58:12.997 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T20:58:12.997 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T20:58:12.997 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T20:58:12.997 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T20:58:12.997 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T20:58:12.998 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T20:58:12.998 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T20:58:12.998 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T20:58:12.998 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T20:58:12.998 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T20:58:12.998 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T20:58:12.998 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T20:58:12.998 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T20:58:12.998 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T20:58:12.998 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T20:58:12.998 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T20:58:12.998 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T20:58:12.998 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T20:58:12.998 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T20:58:12.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T20:58:12.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T20:58:12.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T20:58:12.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T20:58:12.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T20:58:12.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T20:58:12.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T20:58:12.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T20:58:12.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T20:58:12.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T20:58:12.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T20:58:12.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T20:58:12.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T20:58:12.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T20:58:12.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T20:58:12.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T20:58:13.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T20:58:13.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T20:58:13.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T20:58:13.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T20:58:13.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T20:58:13.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T20:58:13.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T20:58:13.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T20:58:13.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T20:58:13.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T20:58:13.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T20:58:13.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T20:58:13.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T20:58:13.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T20:58:13.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T20:58:13.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T20:58:13.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T20:58:13.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T20:58:13.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T20:58:13.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T20:58:13.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T20:58:13.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T20:58:13.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T20:58:13.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T20:58:13.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T20:58:13.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T20:58:13.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T20:58:13.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T20:58:13.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T20:58:13.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T20:58:13.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T20:58:13.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T20:58:13.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T20:58:13.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T20:58:13.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T20:58:13.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T20:58:13.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T20:58:13.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T20:58:13.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T20:58:13.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T20:58:13.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T20:58:13.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T20:58:13.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T20:58:13.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T20:58:13.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T20:58:13.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T20:58:13.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T20:58:13.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T20:58:13.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T20:58:13.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T20:58:13.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T20:58:13.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T20:58:13.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T20:58:13.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T20:58:13.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T20:58:13.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T20:58:13.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T20:58:13.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T20:58:13.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T20:58:13.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T20:58:13.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T20:58:13.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T20:58:13.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T20:58:13.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T20:58:13.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T20:58:13.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T20:58:13.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T20:58:13.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T20:58:13.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T20:58:13.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T20:58:13.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T20:58:13.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T20:58:13.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T20:58:13.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T20:58:13.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T20:58:13.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T20:58:13.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T20:58:13.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T20:58:13.005 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T20:58:13.005 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T20:58:13.005 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T20:58:13.005 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T20:58:13.005 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T20:58:13.005 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T20:58:13.005 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T20:58:13.005 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T20:58:13.005 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T20:58:13.005 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T20:58:13.005 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T20:58:13.005 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T20:58:13.005 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T20:58:13.005 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T20:58:13.006 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T20:58:13.006 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T20:58:13.006 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T20:58:13.006 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T20:58:13.006 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T20:58:13.006 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T20:58:13.006 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T20:58:13.006 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T20:58:13.006 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T20:58:13.006 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T20:58:13.006 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T20:58:13.006 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T20:58:13.006 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T20:58:13.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T20:58:13.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T20:58:13.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T20:58:13.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T20:58:13.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T20:58:13.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T20:58:13.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T20:58:13.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T20:58:13.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T20:58:13.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T20:58:13.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T20:58:13.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T20:58:13.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T20:58:13.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T20:58:13.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T20:58:13.008 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T20:58:13.008 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T20:58:13.008 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T20:58:13.008 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T20:58:13.008 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T20:58:13.008 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T20:58:13.008 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T20:58:13.008 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T20:58:13.008 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T20:58:13.008 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T20:58:13.008 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T20:58:13.008 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T20:58:13.008 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T20:58:13.008 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T20:58:13.008 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T20:58:13.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T20:58:13.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T20:58:13.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T20:58:13.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T20:58:13.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T20:58:13.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T20:58:13.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T20:58:13.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T20:58:13.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T20:58:13.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T20:58:13.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T20:58:13.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T20:58:13.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T20:58:13.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T20:58:13.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T20:58:13.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T20:58:13.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T20:58:13.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T20:58:13.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T20:58:13.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T20:58:13.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T20:58:13.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T20:58:13.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T20:58:13.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T20:58:13.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T20:58:13.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T20:58:13.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T20:58:13.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T20:58:13.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T20:58:13.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T20:58:13.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T20:58:13.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T20:58:13.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T20:58:13.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T20:58:13.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T20:58:13.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T20:58:13.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T20:58:13.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T20:58:13.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T20:58:13.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T20:58:13.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T20:58:13.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T20:58:13.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T20:58:13.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T20:58:13.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T20:58:13.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T20:58:13.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T20:58:13.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T20:58:13.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T20:58:13.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T20:58:13.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T20:58:13.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T20:58:13.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T20:58:13.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T20:58:13.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T20:58:13.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T20:58:13.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T20:58:13.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T20:58:13.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T20:58:13.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T20:58:13.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T20:58:13.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T20:58:13.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T20:58:13.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T20:58:13.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T20:58:13.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T20:58:13.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T20:58:13.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T20:58:13.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T20:58:13.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T20:58:13.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T20:58:13.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T20:58:13.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T20:58:13.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T20:58:13.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T20:58:13.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T20:58:13.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T20:58:13.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T20:58:13.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T20:58:13.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T20:58:13.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T20:58:13.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T20:58:13.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T20:58:13.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T20:58:13.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T20:58:13.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T20:58:13.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T20:58:13.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T20:58:13.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T20:58:13.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T20:58:13.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T20:58:13.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T20:58:13.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T20:58:13.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T20:58:13.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T20:58:13.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T20:58:13.015 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T20:58:13.015 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T20:58:13.015 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T20:58:13.015 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T20:58:13.015 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T20:58:13.015 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T20:58:13.015 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T20:58:13.015 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T20:58:13.015 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T20:58:13.015 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T20:58:13.015 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T20:58:13.015 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T20:58:13.015 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T20:58:13.017 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T20:58:13.017 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T20:58:13.017 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T20:58:13.017 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T20:58:13.017 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T20:58:13.017 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T20:58:13.017 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T20:58:13.017 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T20:58:13.017 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T20:58:13.018 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T20:58:13.018 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T20:58:13.018 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T20:58:13.018 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T20:58:13.018 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T20:58:13.018 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T20:58:13.018 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T20:58:13.018 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T20:58:13.018 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T20:58:13.018 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T20:58:13.018 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T20:58:13.018 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T20:58:13.018 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T20:58:13.018 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T20:58:13.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T20:58:13.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T20:58:13.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T20:58:13.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T20:58:13.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T20:58:13.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T20:58:13.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T20:58:13.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T20:58:13.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T20:58:13.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T20:58:13.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T20:58:13.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T20:58:13.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T20:58:13.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T20:58:13.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T20:58:13.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T20:58:13.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T20:58:13.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T20:58:13.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T20:58:13.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T20:58:13.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T20:58:13.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T20:58:13.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T20:58:13.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T20:58:13.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T20:58:13.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T20:58:13.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T20:58:13.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T20:58:13.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T20:58:13.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T20:58:13.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T20:58:13.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T20:58:13.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T20:58:13.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T20:58:13.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T20:58:13.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T20:58:13.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T20:58:13.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T20:58:13.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T20:58:13.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T20:58:13.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T20:58:13.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T20:58:13.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T20:58:13.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T20:58:13.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T20:58:13.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T20:58:13.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T20:58:13.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T20:58:13.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T20:58:13.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T20:58:13.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T20:58:13.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T20:58:13.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T20:58:13.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T20:58:13.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T20:58:13.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T20:58:13.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T20:58:13.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T20:58:13.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T20:58:13.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T20:58:13.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T20:58:13.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T20:58:13.023 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T20:58:13.023 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T20:58:13.023 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T20:58:13.023 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T20:58:13.023 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T20:58:13.023 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T20:58:13.023 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T20:58:13.023 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T20:58:13.023 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T20:58:13.023 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T20:58:13.023 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T20:58:13.023 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T20:58:13.023 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T20:58:13.023 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T20:58:13.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T20:58:13.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T20:58:13.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T20:58:13.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T20:58:13.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T20:58:13.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T20:58:13.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T20:58:13.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T20:58:13.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T20:58:13.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T20:58:13.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T20:58:13.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T20:58:13.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T20:58:13.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T20:58:13.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T20:58:13.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T20:58:13.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T20:58:13.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T20:58:13.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T20:58:13.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T20:58:13.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T20:58:13.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T20:58:13.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T20:58:13.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T20:58:13.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T20:58:13.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T20:58:13.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T20:58:13.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T20:58:13.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T20:58:13.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T20:58:13.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T20:58:13.026 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T20:58:13.026 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T20:58:13.026 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T20:58:13.026 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T20:58:13.026 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T20:58:13.026 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T20:58:13.026 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T20:58:13.026 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T20:58:13.026 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T20:58:13.026 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T20:58:13.026 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T20:58:13.026 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T20:58:13.026 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T20:58:13.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T20:58:13.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T20:58:13.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T20:58:13.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T20:58:13.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T20:58:13.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T20:58:13.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T20:58:13.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T20:58:13.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T20:58:13.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T20:58:13.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T20:58:13.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T20:58:13.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T20:58:13.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T20:58:13.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T20:58:13.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T20:58:13.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T20:58:13.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T20:58:13.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T20:58:13.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T20:58:13.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T20:58:13.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T20:58:13.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T20:58:13.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T20:58:13.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T20:58:13.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T20:58:13.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T20:58:13.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T20:58:13.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T20:58:13.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T20:58:13.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T20:58:13.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T20:58:13.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T20:58:13.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T20:58:13.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T20:58:13.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T20:58:13.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T20:58:13.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T20:58:13.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T20:58:13.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T20:58:13.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T20:58:13.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T20:58:13.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T20:58:13.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T20:58:13.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T20:58:13.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T20:58:13.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T20:58:13.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T20:58:13.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T20:58:13.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T20:58:13.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T20:58:13.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T20:58:13.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T20:58:13.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T20:58:13.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T20:58:13.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T20:58:13.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T20:58:13.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T20:58:13.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T20:58:13.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T20:58:13.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T20:58:13.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T20:58:13.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T20:58:13.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T20:58:13.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T20:58:13.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T20:58:13.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T20:58:13.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T20:58:13.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T20:58:13.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T20:58:13.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T20:58:13.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T20:58:13.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T20:58:13.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T20:58:13.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T20:58:13.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T20:58:13.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T20:58:13.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T20:58:13.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T20:58:13.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T20:58:13.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T20:58:13.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T20:58:13.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T20:58:13.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T20:58:13.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T20:58:13.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T20:58:13.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T20:58:13.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T20:58:13.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T20:58:13.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T20:58:13.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T20:58:13.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T20:58:13.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T20:58:13.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T20:58:13.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T20:58:13.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T20:58:13.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T20:58:13.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T20:58:13.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T20:58:13.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T20:58:13.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T20:58:13.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T20:58:13.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T20:58:13.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T20:58:13.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T20:58:13.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T20:58:13.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T20:58:13.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T20:58:13.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T20:58:13.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T20:58:13.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T20:58:13.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T20:58:13.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T20:58:13.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T20:58:13.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T20:58:13.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T20:58:13.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T20:58:13.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T20:58:13.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T20:58:13.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T20:58:13.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T20:58:13.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T20:58:13.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T20:58:13.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T20:58:13.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T20:58:13.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T20:58:13.035 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T20:58:13.035 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T20:58:13.035 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T20:58:13.035 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T20:58:13.035 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T20:58:13.035 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T20:58:13.035 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T20:58:13.035 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T20:58:13.035 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T20:58:13.035 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T20:58:13.035 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T20:58:13.035 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T20:58:13.036 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T20:58:13.036 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T20:58:13.036 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T20:58:13.036 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T20:58:13.036 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T20:58:13.036 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T20:58:13.036 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T20:58:13.036 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T20:58:13.036 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T20:58:13.036 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T20:58:13.036 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T20:58:13.036 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T20:58:13.036 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T20:58:13.036 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T20:58:13.037 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T20:58:13.037 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T20:58:13.037 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T20:58:13.037 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T20:58:13.037 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T20:58:13.037 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T20:58:13.037 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T20:58:13.037 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T20:58:13.037 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T20:58:13.037 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T20:58:13.037 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T20:58:13.037 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T20:58:13.037 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T20:58:13.037 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T20:58:13.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T20:58:13.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T20:58:13.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T20:58:13.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T20:58:13.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T20:58:13.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T20:58:13.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T20:58:13.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T20:58:13.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T20:58:13.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T20:58:13.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T20:58:13.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T20:58:13.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T20:58:13.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T20:58:13.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T20:58:13.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T20:58:13.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T20:58:13.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T20:58:13.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T20:58:13.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T20:58:13.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T20:58:13.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T20:58:13.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T20:58:13.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T20:58:13.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T20:58:13.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T20:58:13.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T20:58:13.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T20:58:13.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T20:58:13.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T20:58:13.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T20:58:13.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T20:58:13.040 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T20:58:13.040 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T20:58:13.040 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T20:58:13.040 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T20:58:13.040 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T20:58:13.040 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T20:58:13.040 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T20:58:13.040 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T20:58:13.040 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T20:58:13.040 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T20:58:13.040 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T20:58:13.040 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T20:58:13.040 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T20:58:13.040 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T20:58:13.041 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T20:58:13.041 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T20:58:13.041 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T20:58:13.041 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T20:58:13.041 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T20:58:13.041 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T20:58:13.041 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T20:58:13.041 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T20:58:13.041 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T20:58:13.041 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T20:58:13.041 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T20:58:13.041 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T20:58:13.041 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T20:58:13.041 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T20:58:13.042 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T20:58:13.042 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T20:58:13.042 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T20:58:13.042 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T20:58:13.042 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T20:58:13.042 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T20:58:13.042 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T20:58:13.042 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T20:58:13.042 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T20:58:13.042 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T20:58:13.042 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T20:58:13.042 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T20:58:13.042 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T20:58:13.042 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T20:58:13.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T20:58:13.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T20:58:13.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T20:58:13.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T20:58:13.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T20:58:13.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T20:58:13.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T20:58:13.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T20:58:13.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T20:58:13.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T20:58:13.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T20:58:13.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T20:58:13.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T20:58:13.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T20:58:13.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T20:58:13.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T20:58:13.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T20:58:13.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T20:58:13.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T20:58:13.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T20:58:13.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T20:58:13.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T20:58:13.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T20:58:13.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T20:58:13.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T20:58:13.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T20:58:13.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T20:58:13.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T20:58:13.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T20:58:13.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T20:58:13.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T20:58:13.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T20:58:13.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T20:58:13.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T20:58:13.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T20:58:13.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T20:58:13.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T20:58:13.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T20:58:13.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T20:58:13.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T20:58:13.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T20:58:13.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T20:58:13.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T20:58:13.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T20:58:13.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T20:58:13.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T20:58:13.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T20:58:13.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T20:58:13.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T20:58:13.046 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T20:58:13.046 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T20:58:13.046 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T20:58:13.046 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T20:58:13.046 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T20:58:13.046 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T20:58:13.046 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T20:58:13.046 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T20:58:13.046 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T20:58:13.046 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T20:58:13.046 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T20:58:13.046 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T20:58:13.046 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T20:58:13.046 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T20:58:13.047 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T20:58:13.047 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T20:58:13.047 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T20:58:13.047 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T20:58:13.047 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T20:58:13.047 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T20:58:13.047 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T20:58:13.047 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T20:58:13.047 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T20:58:13.047 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T20:58:13.047 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T20:58:13.047 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T20:58:13.047 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T20:58:13.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T20:58:13.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T20:58:13.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T20:58:13.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T20:58:13.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T20:58:13.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T20:58:13.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T20:58:13.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T20:58:13.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T20:58:13.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T20:58:13.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T20:58:13.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T20:58:13.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T20:58:13.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T20:58:13.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T20:58:13.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T20:58:13.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T20:58:13.049 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T20:58:13.049 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T20:58:13.049 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T20:58:13.049 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T20:58:13.049 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T20:58:13.049 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T20:58:13.049 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T20:58:13.049 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T20:58:13.049 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T20:58:13.049 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T20:58:13.049 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T20:58:13.049 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T20:58:13.049 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T20:58:13.049 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T20:58:13.050 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T20:58:13.050 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T20:58:13.050 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T20:58:13.050 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T20:58:13.050 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T20:58:13.050 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T20:58:13.050 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T20:58:13.050 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T20:58:13.050 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T20:58:13.050 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T20:58:13.050 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T20:58:13.050 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T20:58:13.050 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T20:58:13.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T20:58:13.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T20:58:13.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T20:58:13.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T20:58:13.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T20:58:13.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T20:58:13.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T20:58:13.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T20:58:13.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T20:58:13.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T20:58:13.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T20:58:13.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T20:58:13.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T20:58:13.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T20:58:13.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T20:58:13.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T20:58:13.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T20:58:13.052 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T20:58:13.052 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T20:58:13.052 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T20:58:13.052 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T20:58:13.052 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T20:58:13.052 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T20:58:13.052 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T20:58:13.052 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T20:58:13.052 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T20:58:13.052 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T20:58:13.052 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T20:58:13.052 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T20:58:13.052 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T20:58:13.052 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T20:58:13.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T20:58:13.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T20:58:13.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T20:58:13.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T20:58:13.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T20:58:13.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T20:58:13.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T20:58:13.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T20:58:13.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T20:58:13.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T20:58:13.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T20:58:13.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T20:58:13.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T20:58:13.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T20:58:13.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T20:58:13.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T20:58:13.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T20:58:13.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T20:58:13.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T20:58:13.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T20:58:13.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T20:58:13.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T20:58:13.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T20:58:13.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T20:58:13.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T20:58:13.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T20:58:13.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T20:58:13.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T20:58:13.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T20:58:13.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T20:58:13.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T20:58:13.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T20:58:13.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T20:58:13.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T20:58:13.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T20:58:13.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T20:58:13.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T20:58:13.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T20:58:13.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T20:58:13.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T20:58:13.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T20:58:13.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T20:58:13.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T20:58:13.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T20:58:13.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T20:58:13.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T20:58:13.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T20:58:13.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T20:58:13.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T20:58:13.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T20:58:13.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T20:58:13.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T20:58:13.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T20:58:13.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T20:58:13.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T20:58:13.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T20:58:13.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T20:58:13.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T20:58:13.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T20:58:13.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T20:58:13.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T20:58:13.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T20:58:13.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T20:58:13.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T20:58:13.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T20:58:13.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T20:58:13.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T20:58:13.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T20:58:13.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T20:58:13.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T20:58:13.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T20:58:13.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T20:58:13.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T20:58:13.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T20:58:13.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T20:58:13.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T20:58:13.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T20:58:13.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T20:58:13.058 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T20:58:13.058 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T20:58:13.058 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T20:58:13.058 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T20:58:13.058 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T20:58:13.058 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T20:58:13.058 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T20:58:13.058 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T20:58:13.058 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T20:58:13.058 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T20:58:13.058 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T20:58:13.058 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T20:58:13.058 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T20:58:13.058 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T20:58:13.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T20:58:13.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T20:58:13.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T20:58:13.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T20:58:13.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T20:58:13.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T20:58:13.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T20:58:13.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T20:58:13.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T20:58:13.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T20:58:13.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T20:58:13.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T20:58:13.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T20:58:13.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T20:58:13.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T20:58:13.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T20:58:13.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T20:58:13.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T20:58:13.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T20:58:13.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T20:58:13.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T20:58:13.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T20:58:13.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T20:58:13.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T20:58:13.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T20:58:13.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T20:58:13.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T20:58:13.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T20:58:13.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T20:58:13.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T20:58:13.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T20:58:13.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T20:58:13.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T20:58:13.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T20:58:13.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T20:58:13.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T20:58:13.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T20:58:13.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T20:58:13.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T20:58:13.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T20:58:13.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T20:58:13.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T20:58:13.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T20:58:13.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T20:58:13.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T20:58:13.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T20:58:13.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T20:58:13.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T20:58:13.062 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T20:58:13.062 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T20:58:13.062 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T20:58:13.062 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T20:58:13.062 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T20:58:13.062 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T20:58:13.062 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T20:58:13.062 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T20:58:13.062 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T20:58:13.062 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T20:58:13.062 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T20:58:13.062 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T20:58:13.062 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T20:58:13.063 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T20:58:13.063 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T20:58:13.063 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T20:58:13.063 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T20:58:13.063 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T20:58:13.063 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T20:58:13.063 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T20:58:13.063 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T20:58:13.063 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T20:58:13.063 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T20:58:13.063 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T20:58:13.063 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T20:58:13.063 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T20:58:13.063 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T20:58:13.064 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T20:58:13.064 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T20:58:13.064 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T20:58:13.064 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T20:58:13.064 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T20:58:13.064 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T20:58:13.064 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T20:58:13.064 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T20:58:13.064 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T20:58:13.064 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T20:58:13.064 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T20:58:13.064 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T20:58:13.065 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T20:58:13.065 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T20:58:13.065 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T20:58:13.065 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T20:58:13.065 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T20:58:13.065 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T20:58:13.065 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T20:58:13.065 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T20:58:13.065 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T20:58:13.065 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T20:58:13.065 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T20:58:13.065 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T20:58:13.066 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T20:58:13.066 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T20:58:13.066 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T20:58:13.066 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T20:58:13.066 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T20:58:13.066 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T20:58:13.066 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T20:58:13.066 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T20:58:13.066 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T20:58:13.066 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T20:58:13.066 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T20:58:13.066 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T20:58:13.066 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T20:58:13.066 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T20:58:13.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T20:58:13.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T20:58:13.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T20:58:13.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T20:58:13.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T20:58:13.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T20:58:13.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T20:58:13.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T20:58:13.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T20:58:13.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T20:58:13.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T20:58:13.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T20:58:13.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T20:58:13.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T20:58:13.068 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T20:58:13.068 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T20:58:13.068 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T20:58:13.068 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T20:58:13.068 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T20:58:13.068 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T20:58:13.068 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T20:58:13.068 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T20:58:13.068 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T20:58:13.068 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T20:58:13.068 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T20:58:13.068 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T20:58:13.068 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T20:58:13.068 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T20:58:13.068 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T20:58:13.069 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T20:58:13.069 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T20:58:13.069 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T20:58:13.069 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T20:58:13.069 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T20:58:13.069 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T20:58:13.069 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T20:58:13.069 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T20:58:13.069 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T20:58:13.069 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T20:58:13.069 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T20:58:13.069 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T20:58:13.070 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T20:58:13.070 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T20:58:13.070 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T20:58:13.070 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T20:58:13.070 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T20:58:13.070 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T20:58:13.070 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T20:58:13.070 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T20:58:13.070 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T20:58:13.070 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T20:58:13.070 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T20:58:13.070 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T20:58:13.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T20:58:13.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T20:58:13.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T20:58:13.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T20:58:13.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T20:58:13.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T20:58:13.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T20:58:13.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T20:58:13.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T20:58:13.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T20:58:13.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T20:58:13.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T20:58:13.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T20:58:13.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T20:58:13.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T20:58:13.072 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T20:58:13.072 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T20:58:13.072 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T20:58:13.072 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T20:58:13.072 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T20:58:13.072 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T20:58:13.072 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T20:58:13.072 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T20:58:13.072 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T20:58:13.072 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T20:58:13.072 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T20:58:13.072 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T20:58:13.072 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T20:58:13.072 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T20:58:13.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T20:58:13.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T20:58:13.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T20:58:13.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T20:58:13.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T20:58:13.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T20:58:13.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T20:58:13.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T20:58:13.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T20:58:13.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T20:58:13.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T20:58:13.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T20:58:13.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T20:58:13.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T20:58:13.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T20:58:13.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T20:58:13.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T20:58:13.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T20:58:13.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T20:58:13.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T20:58:13.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T20:58:13.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T20:58:13.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T20:58:13.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T20:58:13.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T20:58:13.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T20:58:13.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T20:58:13.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T20:58:13.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T20:58:13.076 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T20:58:13.076 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T20:58:13.076 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T20:58:13.076 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T20:58:13.076 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T20:58:13.076 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T20:58:13.076 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T20:58:13.076 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T20:58:13.076 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T20:58:13.076 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T20:58:13.076 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T20:58:13.076 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T20:58:13.076 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T20:58:13.076 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T20:58:13.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T20:58:13.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T20:58:13.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T20:58:13.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T20:58:13.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T20:58:13.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T20:58:13.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T20:58:13.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T20:58:13.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T20:58:13.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T20:58:13.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T20:58:13.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T20:58:13.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T20:58:13.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T20:58:13.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T20:58:13.079 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T20:58:13.079 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T20:58:13.079 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T20:58:13.079 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T20:58:13.079 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T20:58:13.079 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T20:58:13.079 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T20:58:13.079 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T20:58:13.079 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T20:58:13.079 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T20:58:13.079 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T20:58:13.079 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T20:58:13.079 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T20:58:13.079 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T20:58:13.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T20:58:13.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T20:58:13.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T20:58:13.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T20:58:13.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T20:58:13.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T20:58:13.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T20:58:13.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T20:58:13.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T20:58:13.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T20:58:13.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T20:58:13.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T20:58:13.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T20:58:13.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T20:58:13.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T20:58:13.082 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T20:58:13.082 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T20:58:13.082 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T20:58:13.082 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T20:58:13.082 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T20:58:13.082 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T20:58:13.082 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T20:58:13.082 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T20:58:13.082 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T20:58:13.082 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T20:58:13.082 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T20:58:13.082 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T20:58:13.083 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T20:58:13.083 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T20:58:13.083 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T20:58:13.083 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T20:58:13.083 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T20:58:13.083 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T20:58:13.083 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T20:58:13.083 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T20:58:13.083 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T20:58:13.083 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T20:58:13.083 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T20:58:13.083 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T20:58:13.083 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T20:58:13.083 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T20:58:13.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T20:58:13.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T20:58:13.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T20:58:13.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T20:58:13.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T20:58:13.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T20:58:13.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T20:58:13.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T20:58:13.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T20:58:13.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T20:58:13.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T20:58:13.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T20:58:13.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T20:58:13.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T20:58:13.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T20:58:13.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T20:58:13.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T20:58:13.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T20:58:13.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T20:58:13.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T20:58:13.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T20:58:13.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T20:58:13.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T20:58:13.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T20:58:13.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T20:58:13.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T20:58:13.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T20:58:13.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T20:58:13.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T20:58:13.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T20:58:13.086 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T20:58:13.086 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T20:58:13.086 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T20:58:13.086 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T20:58:13.086 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T20:58:13.086 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T20:58:13.086 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T20:58:13.086 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T20:58:13.086 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T20:58:13.086 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T20:58:13.086 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T20:58:13.086 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T20:58:13.086 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T20:58:13.086 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T20:58:13.087 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T20:58:13.087 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T20:58:13.087 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T20:58:13.087 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T20:58:13.087 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T20:58:13.087 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T20:58:13.087 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T20:58:13.087 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T20:58:13.087 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T20:58:13.087 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T20:58:13.087 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T20:58:13.087 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T20:58:13.087 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T20:58:13.087 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T20:58:13.088 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T20:58:13.088 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T20:58:13.088 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T20:58:13.088 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T20:58:13.088 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T20:58:13.088 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T20:58:13.088 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T20:58:13.088 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T20:58:13.088 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T20:58:13.088 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T20:58:13.088 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T20:58:13.088 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T20:58:13.088 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T20:58:13.088 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T20:58:13.089 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T20:58:13.089 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T20:58:13.089 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T20:58:13.089 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T20:58:13.089 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T20:58:13.089 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T20:58:13.089 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T20:58:13.089 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T20:58:13.089 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T20:58:13.089 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T20:58:13.089 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T20:58:13.089 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T20:58:13.089 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T20:58:13.090 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T20:58:13.090 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T20:58:13.090 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T20:58:13.090 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T20:58:13.090 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T20:58:13.090 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T20:58:13.090 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T20:58:13.090 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T20:58:13.090 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T20:58:13.090 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T20:58:13.090 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T20:58:13.090 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T20:58:13.090 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T20:58:13.090 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T20:58:13.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T20:58:13.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T20:58:13.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T20:58:13.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T20:58:13.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T20:58:13.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T20:58:13.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T20:58:13.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T20:58:13.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T20:58:13.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T20:58:13.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T20:58:13.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T20:58:13.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T20:58:13.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T20:58:13.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T20:58:13.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T20:58:13.092 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T20:58:13.092 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T20:58:13.092 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T20:58:13.092 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T20:58:13.092 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T20:58:13.092 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T20:58:13.092 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T20:58:13.092 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T20:58:13.092 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T20:58:13.092 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T20:58:13.092 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T20:58:13.092 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T20:58:13.092 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T20:58:13.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T20:58:13.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T20:58:13.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T20:58:13.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T20:58:13.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T20:58:13.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T20:58:13.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T20:58:13.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T20:58:13.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T20:58:13.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T20:58:13.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T20:58:13.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T20:58:13.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T20:58:13.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T20:58:13.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T20:58:13.094 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T20:58:13.094 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T20:58:13.094 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T20:58:13.094 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T20:58:13.094 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T20:58:13.094 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T20:58:13.094 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T20:58:13.094 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T20:58:13.094 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T20:58:13.094 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T20:58:13.094 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T20:58:13.094 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T20:58:13.094 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T20:58:13.094 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T20:58:13.095 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T20:58:13.095 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T20:58:13.095 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T20:58:13.095 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T20:58:13.095 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T20:58:13.095 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T20:58:13.095 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T20:58:13.095 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T20:58:13.095 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T20:58:13.095 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T20:58:13.095 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T20:58:13.095 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T20:58:13.095 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T20:58:13.095 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T20:58:13.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T20:58:13.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T20:58:13.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T20:58:13.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T20:58:13.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T20:58:13.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T20:58:13.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T20:58:13.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T20:58:13.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T20:58:13.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T20:58:13.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T20:58:13.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T20:58:13.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T20:58:13.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T20:58:13.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T20:58:13.097 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T20:58:13.097 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T20:58:13.097 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T20:58:13.097 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T20:58:13.097 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T20:58:13.097 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T20:58:13.097 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T20:58:13.097 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T20:58:13.097 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T20:58:13.097 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T20:58:13.097 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T20:58:13.097 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T20:58:13.097 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T20:58:13.097 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T20:58:13.098 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T20:58:13.098 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T20:58:13.098 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T20:58:13.098 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T20:58:13.098 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T20:58:13.098 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T20:58:13.098 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T20:58:13.098 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T20:58:13.098 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T20:58:13.098 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T20:58:13.098 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T20:58:13.098 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T20:58:13.098 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T20:58:13.098 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T20:58:13.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T20:58:13.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T20:58:13.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T20:58:13.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T20:58:13.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T20:58:13.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T20:58:13.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T20:58:13.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T20:58:13.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T20:58:13.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T20:58:13.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T20:58:13.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T20:58:13.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T20:58:13.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T20:58:13.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T20:58:13.100 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T20:58:13.100 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T20:58:13.100 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T20:58:13.100 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T20:58:13.100 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T20:58:13.100 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T20:58:13.100 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T20:58:13.100 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T20:58:13.100 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T20:58:13.100 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T20:58:13.100 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T20:58:13.100 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T20:58:13.100 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T20:58:13.100 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T20:58:13.100 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T20:58:13.101 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T20:58:13.101 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T20:58:13.101 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T20:58:13.101 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T20:58:13.101 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T20:58:13.101 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T20:58:13.101 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T20:58:13.101 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T20:58:13.101 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T20:58:13.101 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T20:58:13.101 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T20:58:13.101 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T20:58:13.101 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T20:58:13.101 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T20:58:13.101 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T20:58:13.101 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T20:58:13.102 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T20:58:13.102 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T20:58:13.102 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T20:58:13.102 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T20:58:13.102 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T20:58:13.102 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T20:58:13.102 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T20:58:13.102 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T20:58:13.102 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T20:58:13.102 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T20:58:13.102 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T20:58:13.102 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T20:58:13.102 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T20:58:13.102 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T20:58:13.102 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T20:58:13.103 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T20:58:13.103 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T20:58:13.103 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T20:58:13.103 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T20:58:13.103 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T20:58:13.103 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T20:58:13.103 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T20:58:13.103 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T20:58:13.103 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T20:58:13.103 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T20:58:13.103 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T20:58:13.103 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T20:58:13.103 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T20:58:13.103 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T20:58:13.103 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T20:58:13.103 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T20:58:13.104 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T20:58:13.104 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T20:58:13.104 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T20:58:13.104 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T20:58:13.104 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T20:58:13.104 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T20:58:13.104 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T20:58:13.104 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T20:58:13.104 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T20:58:13.104 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T20:58:13.104 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T20:58:13.104 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T20:58:13.104 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T20:58:13.104 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T20:58:13.104 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T20:58:13.104 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T20:58:13.104 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T20:58:13.105 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T20:58:13.105 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T20:58:13.105 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T20:58:13.105 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T20:58:13.105 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T20:58:13.105 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T20:58:13.105 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T20:58:13.105 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T20:58:13.105 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T20:58:13.105 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T20:58:13.105 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T20:58:13.105 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T20:58:13.105 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T20:58:13.105 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T20:58:13.105 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T20:58:13.105 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T20:58:13.105 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T20:58:13.106 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T20:58:13.106 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T20:58:13.106 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T20:58:13.106 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T20:58:13.106 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T20:58:13.106 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T20:58:13.106 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T20:58:13.106 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T20:58:13.106 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T20:58:13.106 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T20:58:13.106 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T20:58:13.106 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T20:58:13.107 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T20:58:13.107 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T20:58:13.107 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T20:58:13.107 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T20:58:13.107 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T20:58:13.107 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T20:58:13.107 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T20:58:13.107 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T20:58:13.107 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T20:58:13.107 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T20:58:13.107 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T20:58:13.107 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T20:58:13.107 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T20:58:13.107 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T20:58:13.107 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T20:58:13.107 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T20:58:13.108 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T20:58:13.108 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T20:58:13.108 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T20:58:13.108 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T20:58:13.108 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T20:58:13.108 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T20:58:13.108 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T20:58:13.108 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T20:58:13.108 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T20:58:13.108 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T20:58:13.108 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T20:58:13.108 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T20:58:13.108 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T20:58:13.108 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T20:58:13.109 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T20:58:13.109 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T20:58:13.109 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T20:58:13.109 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T20:58:13.109 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T20:58:13.109 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T20:58:13.109 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T20:58:13.109 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T20:58:13.109 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T20:58:13.109 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T20:58:13.109 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T20:58:13.109 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T20:58:13.109 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T20:58:13.110 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T20:58:13.110 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T20:58:13.110 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T20:58:13.110 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T20:58:13.110 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T20:58:13.110 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T20:58:13.110 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T20:58:13.110 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T20:58:13.110 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T20:58:13.110 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T20:58:13.110 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T20:58:13.110 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T20:58:13.110 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T20:58:13.110 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T20:58:13.110 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T20:58:13.111 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T20:58:13.111 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T20:58:13.111 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T20:58:13.111 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T20:58:13.111 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T20:58:13.111 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T20:58:13.111 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T20:58:13.111 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T20:58:13.111 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T20:58:13.111 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T20:58:13.111 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T20:58:13.111 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T20:58:13.111 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T20:58:13.111 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T20:58:13.112 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T20:58:13.112 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T20:58:13.112 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T20:58:13.112 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T20:58:13.112 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T20:58:13.112 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T20:58:13.112 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T20:58:13.112 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T20:58:13.112 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T20:58:13.112 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T20:58:13.112 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T20:58:13.112 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T20:58:13.112 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T20:58:13.112 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T20:58:13.112 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T20:58:13.112 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T20:58:13.113 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T20:58:13.113 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T20:58:13.113 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T20:58:13.113 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T20:58:13.113 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T20:58:13.113 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T20:58:13.113 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T20:58:13.113 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T20:58:13.113 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T20:58:13.113 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T20:58:13.113 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T20:58:13.113 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T20:58:13.113 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T20:58:13.113 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T20:58:13.113 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T20:58:13.113 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T20:58:13.114 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T20:58:13.114 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T20:58:13.114 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T20:58:13.114 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T20:58:13.114 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T20:58:13.114 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T20:58:13.114 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T20:58:13.114 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T20:58:13.114 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T20:58:13.114 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T20:58:13.114 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T20:58:13.114 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T20:58:13.114 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T20:58:13.114 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T20:58:13.114 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T20:58:13.115 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T20:58:13.115 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T20:58:13.115 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T20:58:13.115 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T20:58:13.115 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T20:58:13.115 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T20:58:13.115 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T20:58:13.115 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T20:58:13.115 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T20:58:13.115 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T20:58:13.115 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T20:58:13.115 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T20:58:13.115 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T20:58:13.115 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T20:58:13.115 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T20:58:13.115 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T20:58:13.115 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T20:58:13.116 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T20:58:13.116 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T20:58:13.116 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T20:58:13.116 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T20:58:13.116 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T20:58:13.116 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T20:58:13.116 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T20:58:13.116 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T20:58:13.116 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T20:58:13.116 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T20:58:13.116 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T20:58:13.116 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T20:58:13.116 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T20:58:13.116 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T20:58:13.116 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T20:58:13.116 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T20:58:13.117 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T20:58:13.120 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T20:58:13.120 DEBUG:teuthology.orchestra.run.smithi026:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T20:58:13.144 INFO:teuthology.orchestra.run.smithi026.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T20:58:13.144 INFO:teuthology.orchestra.run.smithi026.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T20:58:13.144 INFO:teuthology.orchestra.run.smithi026.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T20:58:13.144 INFO:teuthology.orchestra.run.smithi026.stdout:fi 2024-04-26T20:58:13.144 INFO:teuthology.orchestra.run.smithi026.stdout:for file in vmlinuz System.map config; do 2024-04-26T20:58:13.144 INFO:teuthology.orchestra.run.smithi026.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T20:58:13.144 INFO:teuthology.orchestra.run.smithi026.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:58:13.144 INFO:teuthology.orchestra.run.smithi026.stdout: fi 2024-04-26T20:58:13.144 INFO:teuthology.orchestra.run.smithi026.stdout:done 2024-04-26T20:58:13.144 INFO:teuthology.orchestra.run.smithi026.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T20:58:13.145 INFO:teuthology.orchestra.run.smithi026.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T20:58:13.145 INFO:teuthology.orchestra.run.smithi026.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:58:13.145 INFO:teuthology.orchestra.run.smithi026.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:58:13.145 INFO:teuthology.orchestra.run.smithi026.stdout:fi 2024-04-26T20:58:13.145 INFO:teuthology.orchestra.run.smithi026.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T20:58:13.145 INFO:teuthology.orchestra.run.smithi026.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T20:58:13.145 INFO:teuthology.orchestra.run.smithi026.stdout:fi 2024-04-26T20:58:13.145 INFO:teuthology.orchestra.run.smithi026.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T20:58:13.145 INFO:teuthology.orchestra.run.smithi026.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T20:58:13.145 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T20:58:13.145 INFO:teuthology.orchestra.run.smithi026.stdout:fi 2024-04-26T20:58:13.146 DEBUG:teuthology.orchestra.run.smithi026:> sudo rpm -qi grub2-tools 2024-04-26T20:58:13.218 INFO:teuthology.orchestra.run.smithi026.stdout:Name : grub2-tools 2024-04-26T20:58:13.218 INFO:teuthology.orchestra.run.smithi026.stdout:Epoch : 1 2024-04-26T20:58:13.219 INFO:teuthology.orchestra.run.smithi026.stdout:Version : 2.06 2024-04-26T20:58:13.219 INFO:teuthology.orchestra.run.smithi026.stdout:Release : 68.el9 2024-04-26T20:58:13.219 INFO:teuthology.orchestra.run.smithi026.stdout:Architecture: x86_64 2024-04-26T20:58:13.219 INFO:teuthology.orchestra.run.smithi026.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T20:58:13.219 INFO:teuthology.orchestra.run.smithi026.stdout:Group : Unspecified 2024-04-26T20:58:13.219 INFO:teuthology.orchestra.run.smithi026.stdout:Size : 8264254 2024-04-26T20:58:13.219 INFO:teuthology.orchestra.run.smithi026.stdout:License : GPLv3+ 2024-04-26T20:58:13.219 INFO:teuthology.orchestra.run.smithi026.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T20:58:13.219 INFO:teuthology.orchestra.run.smithi026.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T20:58:13.219 INFO:teuthology.orchestra.run.smithi026.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T20:58:13.219 INFO:teuthology.orchestra.run.smithi026.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T20:58:13.222 INFO:teuthology.orchestra.run.smithi026.stdout:Packager : builder@centos.org 2024-04-26T20:58:13.223 INFO:teuthology.orchestra.run.smithi026.stdout:Vendor : CentOS 2024-04-26T20:58:13.223 INFO:teuthology.orchestra.run.smithi026.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T20:58:13.223 INFO:teuthology.orchestra.run.smithi026.stdout:Summary : Support tools for GRUB. 2024-04-26T20:58:13.223 INFO:teuthology.orchestra.run.smithi026.stdout:Description : 2024-04-26T20:58:13.223 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T20:58:13.223 INFO:teuthology.orchestra.run.smithi026.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T20:58:13.223 INFO:teuthology.orchestra.run.smithi026.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T20:58:13.223 INFO:teuthology.orchestra.run.smithi026.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T20:58:13.223 INFO:teuthology.orchestra.run.smithi026.stdout:hardware devices. 2024-04-26T20:58:13.223 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T20:58:13.223 INFO:teuthology.orchestra.run.smithi026.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T20:58:13.224 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T20:58:13.224 INFO:teuthology.task.kernel:Updating grub on smithi026 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T20:58:13.224 DEBUG:teuthology.orchestra.run.smithi026:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T20:58:13.810 INFO:teuthology.orchestra.run.smithi026.stderr:Generating grub configuration file ... 2024-04-26T20:58:15.252 INFO:teuthology.orchestra.run.smithi026.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T20:58:15.277 INFO:teuthology.orchestra.run.smithi026.stderr:done 2024-04-26T20:58:15.279 DEBUG:teuthology.orchestra.run.smithi026:> mktemp 2024-04-26T20:58:15.304 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/tmp.VvDMDnfy33 2024-04-26T20:58:15.304 DEBUG:teuthology.orchestra.run.smithi026:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.VvDMDnfy33 2024-04-26T20:58:15.369 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0666 /tmp/tmp.VvDMDnfy33 2024-04-26T20:58:15.516 DEBUG:teuthology.orchestra.remote:smithi026:/tmp/tmp.VvDMDnfy33 is 6KB 2024-04-26T20:58:15.553 DEBUG:teuthology.orchestra.run.smithi026:> rm -fr /tmp/tmp.VvDMDnfy33 2024-04-26T20:58:15.568 DEBUG:teuthology.orchestra.run.smithi026:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T20:58:15.643 INFO:teuthology.orchestra.run.smithi026.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T20:58:15.643 INFO:teuthology.orchestra.run.smithi026.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T20:58:15.643 INFO:teuthology.orchestra.run.smithi026.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T20:58:15.643 INFO:teuthology.orchestra.run.smithi026.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T20:58:15.644 DEBUG:teuthology.orchestra.run.smithi026:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T20:58:15.791 DEBUG:teuthology.orchestra.run.smithi026:> sudo shutdown -r now 2024-04-26T20:58:36.191 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-04-26T20:58:45.273 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-26T20:58:45.275 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:58:45.796 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-26T20:58:45.796 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-04-26T20:58:45.797 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:59:03.838 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-04-26T20:59:15.850 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-26T20:59:15.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:59:18.910 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2024-04-26T20:59:33.922 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-26T20:59:33.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:59:34.273 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-04-26T20:59:34.891 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-26T20:59:34.892 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T20:59:34.892 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-04-26T20:59:34.940 INFO:teuthology.orchestra.run.smithi086.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T20:59:34.941 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T20:59:34.941 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T20:59:34.941 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T20:59:35.942 DEBUG:teuthology.parallel:result is None 2024-04-26T20:59:45.799 DEBUG:teuthology.orchestra.remote:timed out 2024-04-26T20:59:54.801 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-04-26T20:59:54.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:59:55.246 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-04-26T20:59:55.867 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-04-26T20:59:55.868 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T20:59:55.868 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2024-04-26T20:59:55.914 INFO:teuthology.orchestra.run.smithi026.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T20:59:55.914 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T20:59:55.914 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T20:59:55.914 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T20:59:56.917 DEBUG:teuthology.parallel:result is None 2024-04-26T20:59:56.918 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T20:59:56.926 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T20:59:56.926 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T20:59:56.932 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T20:59:56.989 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T20:59:56.997 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T20:59:57.004 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T20:59:57.005 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T20:59:57.016 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T20:59:57.085 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T20:59:57.091 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T20:59:57.092 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T20:59:57.096 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T20:59:57.192 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:59:57.254 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:59:57.256 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:59:57.309 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:59:57.314 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T20:59:57.322 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T20:59:57.322 DEBUG:teuthology.orchestra.run.smithi026:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T20:59:57.326 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T20:59:57.417 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T20:59:57.426 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T20:59:57.426 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T20:59:57.430 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T20:59:57.482 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:59:57.576 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:59:57.664 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:59:57.731 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:59:57.807 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T20:59:57.807 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T20:59:57.865 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:59:57.952 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:59:58.048 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:59:58.135 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:59:58.221 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T20:59:58.221 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T20:59:58.305 DEBUG:teuthology.orchestra.run.smithi026:> sudo service rsyslog restart 2024-04-26T20:59:58.310 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2024-04-26T20:59:58.413 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:59:58.468 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:59:58.821 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T20:59:58.854 INFO:teuthology.task.internal:Starting timer... 2024-04-26T20:59:58.854 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T20:59:58.864 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T20:59:58.871 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-26T20:59:58.872 DEBUG:teuthology.orchestra.run.smithi026:> sudo service auditd rotate 2024-04-26T20:59:59.006 INFO:teuthology.orchestra.run.smithi026.stdout:Rotating logs: 2024-04-26T20:59:59.011 DEBUG:teuthology.orchestra.run.smithi086:> sudo service auditd rotate 2024-04-26T20:59:59.121 INFO:teuthology.orchestra.run.smithi086.stdout:Rotating logs: 2024-04-26T20:59:59.125 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T20:59:59.125 DEBUG:teuthology.orchestra.run.smithi026:> /usr/sbin/getenforce 2024-04-26T20:59:59.190 INFO:teuthology.orchestra.run.smithi026.stdout:Permissive 2024-04-26T20:59:59.191 DEBUG:teuthology.orchestra.run.smithi086:> /usr/sbin/getenforce 2024-04-26T20:59:59.240 INFO:teuthology.orchestra.run.smithi086.stdout:Permissive 2024-04-26T20:59:59.241 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi026.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi086.front.sepia.ceph.com': 'permissive'} 2024-04-26T20:59:59.241 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:59:59.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:59:59.312 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:59:59.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:59:59.394 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T20:59:59.394 DEBUG:teuthology.orchestra.run.smithi026:> sudo /usr/sbin/setenforce permissive 2024-04-26T20:59:59.471 DEBUG:teuthology.orchestra.run.smithi086:> sudo /usr/sbin/setenforce permissive 2024-04-26T20:59:59.565 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T20:59:59.575 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-26T20:59:59.580 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-26T20:59:59.686 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T20:59:59.702 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T20:59:59.703 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi026.front.sepia.ceph.com,smithi086.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T21:05:10.926 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi026.front.sepia.ceph.com'), Remote(name='ubuntu@smithi086.front.sepia.ceph.com')] 2024-04-26T21:05:10.928 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-04-26T21:05:10.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:05:11.061 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-04-26T21:05:11.209 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-04-26T21:05:11.210 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-26T21:05:11.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:05:11.313 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-04-26T21:05:11.478 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-26T21:05:11.479 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T21:05:11.491 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T21:05:11.491 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:05:11.492 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:05:11.500 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:05:11.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:05:11.587 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:05:11.598 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:05:11.628 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:05:11.644 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:05:11.704 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: ntpd: command not found 2024-04-26T21:05:11.722 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: ntpd: command not found 2024-04-26T21:05:11.736 INFO:teuthology.orchestra.run.smithi086.stdout:506 Cannot talk to daemon 2024-04-26T21:05:11.757 INFO:teuthology.orchestra.run.smithi026.stdout:506 Cannot talk to daemon 2024-04-26T21:05:11.778 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:05:11.802 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:05:11.821 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:05:11.848 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:05:11.961 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:05:11.971 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:05:11.971 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-04-26T21:05:11.971 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:05:11.971 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:05:11.971 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:05:11.972 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:05:12.001 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:05:12.010 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:05:12.010 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-04-26T21:05:12.010 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:05:12.010 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:05:12.010 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:05:12.010 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:05:12.011 INFO:teuthology.run_tasks:Running task install... 2024-04-26T21:05:12.021 DEBUG:teuthology.task.install:project ceph 2024-04-26T21:05:12.021 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T21:05:12.021 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T21:05:12.021 INFO:teuthology.task.install:Using flavor: default 2024-04-26T21:05:12.027 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T21:05:12.028 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T21:05:12.028 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:05:12.028 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:05:12.031 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:05:12.031 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:05:12.257 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:05:12.257 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:05:12.273 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:05:12.273 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:05:12.445 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:05:12.445 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:05:12.446 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:05:12.456 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:05:12.456 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:05:12.456 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:05:12.535 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:05:12.535 DEBUG:teuthology.orchestra.run.smithi026:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:05:12.547 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:05:12.547 DEBUG:teuthology.orchestra.run.smithi086:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:05:12.671 DEBUG:teuthology.orchestra.run.smithi026:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:05:12.675 DEBUG:teuthology.orchestra.run.smithi086:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:05:12.787 DEBUG:teuthology.orchestra.run.smithi086:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:05:12.789 DEBUG:teuthology.orchestra.run.smithi026:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:05:12.864 INFO:teuthology.orchestra.run.smithi086.stdout:check_obsoletes = 1 2024-04-26T21:05:12.868 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum clean all 2024-04-26T21:05:12.873 INFO:teuthology.orchestra.run.smithi026.stdout:check_obsoletes = 1 2024-04-26T21:05:12.875 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum clean all 2024-04-26T21:05:13.187 INFO:teuthology.orchestra.run.smithi086.stdout:48 files removed 2024-04-26T21:05:13.209 INFO:teuthology.orchestra.run.smithi026.stdout:48 files removed 2024-04-26T21:05:13.225 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:05:13.249 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:05:14.062 INFO:teuthology.orchestra.run.smithi026.stdout:ceph packages for x86_64 231 kB/s | 84 kB 00:00 2024-04-26T21:05:14.098 INFO:teuthology.orchestra.run.smithi086.stdout:ceph packages for x86_64 196 kB/s | 84 kB 00:00 2024-04-26T21:05:14.563 INFO:teuthology.orchestra.run.smithi026.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-04-26T21:05:14.690 INFO:teuthology.orchestra.run.smithi086.stdout:ceph noarch packages 24 kB/s | 12 kB 00:00 2024-04-26T21:05:15.130 INFO:teuthology.orchestra.run.smithi026.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-04-26T21:05:15.207 INFO:teuthology.orchestra.run.smithi086.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-04-26T21:05:15.720 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-26T21:05:16.389 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 7.3 MB/s | 8.0 MB 00:01 2024-04-26T21:05:19.798 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 8.8 MB/s | 19 MB 00:02 2024-04-26T21:05:25.183 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.2 MB 00:01 2024-04-26T21:05:27.200 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 16 kB 00:00 2024-04-26T21:05:36.036 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 19 MB 00:19 2024-04-26T21:05:40.902 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - CRB 7.2 MB/s | 6.2 MB 00:00 2024-04-26T21:05:42.536 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 16 kB 00:00 2024-04-26T21:05:45.310 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 8.1 MB/s | 21 MB 00:02 2024-04-26T21:05:47.122 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-04-26T21:05:51.513 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-04-26T21:05:53.049 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-26T21:05:53.978 INFO:teuthology.orchestra.run.smithi026.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:05:53.980 INFO:teuthology.orchestra.run.smithi026.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:05:53.999 INFO:teuthology.orchestra.run.smithi026.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:05:54.000 INFO:teuthology.orchestra.run.smithi026.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:05:54.001 INFO:teuthology.orchestra.run.smithi026.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:05:54.002 INFO:teuthology.orchestra.run.smithi026.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:05:54.005 INFO:teuthology.orchestra.run.smithi026.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:05:54.008 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:05:54.009 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:05:54.084 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-04-26T21:05:54.102 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-04-26T21:05:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-04-26T21:05:54.102 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-04-26T21:05:54.102 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-04-26T21:05:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:05:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:05:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:05:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:05:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:05:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:05:54.102 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:05:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:05:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:05:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:05:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:05:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:05:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:05:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:05:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:05:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:05:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:05:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:05:54.103 INFO:teuthology.orchestra.run.smithi026.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:05:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:05:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:05:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:05:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:05:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:05:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:05:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:05:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:05:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:05:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:05:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:05:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:05:54.104 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:05:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:05:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:05:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:05:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:05:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:05:54.105 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrading: 2024-04-26T21:05:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:05:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:05:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:05:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:05:54.105 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-04-26T21:05:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:05:54.105 INFO:teuthology.orchestra.run.smithi026.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:05:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:05:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:05:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:05:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:05:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:05:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:05:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:05:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:05:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:05:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:05:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:05:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:05:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:05:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:05:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:05:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:05:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:05:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:05:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:05:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:05:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:05:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:05:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:05:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:05:54.107 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:05:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:05:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:05:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:05:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:05:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:05:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:05:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:05:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:05:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:05:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:05:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:05:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:05:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:05:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:05:54.108 INFO:teuthology.orchestra.run.smithi026.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:05:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:05:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:05:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:05:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:05:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:05:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:05:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:05:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:05:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:05:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:05:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:05:54.109 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:05:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:05:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:05:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:05:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:05:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:05:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:05:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:05:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:05:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:05:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:05:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:05:54.110 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:05:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:05:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:05:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:05:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:05:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:05:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:05:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:05:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:05:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:05:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:05:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:05:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:05:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:05:54.111 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:05:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:05:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:05:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:05:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:05:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:05:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:05:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:05:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:05:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:05:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:05:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:05:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:05:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:05:54.112 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:05:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:05:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:05:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:05:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:05:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:05:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:05:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:05:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:05:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:05:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:05:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:05:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:05:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:05:54.113 INFO:teuthology.orchestra.run.smithi026.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:05:54.114 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:05:54.114 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:05:54.114 INFO:teuthology.orchestra.run.smithi026.stdout:Installing weak dependencies: 2024-04-26T21:05:54.114 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:05:54.114 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:05:54.114 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-04-26T21:05:54.114 INFO:teuthology.orchestra.run.smithi026.stdout:======================================================================================= 2024-04-26T21:05:54.114 INFO:teuthology.orchestra.run.smithi026.stdout:Install 148 Packages 2024-04-26T21:05:54.114 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrade 4 Packages 2024-04-26T21:05:54.114 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:05:54.115 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 222 M 2024-04-26T21:05:54.115 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-04-26T21:05:55.214 INFO:teuthology.orchestra.run.smithi026.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-04-26T21:05:55.517 INFO:teuthology.orchestra.run.smithi086.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:05:55.519 INFO:teuthology.orchestra.run.smithi086.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:05:55.540 INFO:teuthology.orchestra.run.smithi086.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:05:55.541 INFO:teuthology.orchestra.run.smithi086.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:05:55.542 INFO:teuthology.orchestra.run.smithi086.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:05:55.544 INFO:teuthology.orchestra.run.smithi086.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:05:55.546 INFO:teuthology.orchestra.run.smithi086.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:05:55.549 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:05:55.551 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:05:55.623 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-04-26T21:05:55.645 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-04-26T21:05:55.646 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-04-26T21:05:55.646 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-04-26T21:05:55.646 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-04-26T21:05:55.646 INFO:teuthology.orchestra.run.smithi086.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:05:55.646 INFO:teuthology.orchestra.run.smithi086.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:05:55.646 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:05:55.646 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:05:55.646 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:05:55.646 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:05:55.646 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:05:55.646 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:05:55.647 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:05:55.647 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:05:55.647 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:05:55.647 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:05:55.647 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:05:55.647 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:05:55.647 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:05:55.647 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:05:55.647 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:05:55.647 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:05:55.647 INFO:teuthology.orchestra.run.smithi086.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:05:55.647 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:05:55.647 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:05:55.647 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:05:55.647 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:05:55.648 INFO:teuthology.orchestra.run.smithi086.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:05:55.648 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:05:55.648 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:05:55.648 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:05:55.648 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:05:55.648 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:05:55.648 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:05:55.648 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:05:55.648 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:05:55.648 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:05:55.648 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:05:55.648 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:05:55.648 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:05:55.648 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:05:55.649 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrading: 2024-04-26T21:05:55.649 INFO:teuthology.orchestra.run.smithi086.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:05:55.649 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:05:55.649 INFO:teuthology.orchestra.run.smithi086.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:05:55.649 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:05:55.649 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-04-26T21:05:55.649 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:05:55.649 INFO:teuthology.orchestra.run.smithi086.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:05:55.649 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:05:55.649 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:05:55.649 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:05:55.649 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:05:55.649 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:05:55.649 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:05:55.650 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:05:55.650 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:05:55.650 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:05:55.650 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:05:55.650 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:05:55.650 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:05:55.650 INFO:teuthology.orchestra.run.smithi086.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:05:55.650 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:05:55.650 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:05:55.650 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:05:55.650 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:05:55.650 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:05:55.650 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:05:55.651 INFO:teuthology.orchestra.run.smithi086.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:05:55.651 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:05:55.651 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:05:55.651 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:05:55.651 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:05:55.651 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:05:55.651 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:05:55.651 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:05:55.651 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:05:55.651 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:05:55.651 INFO:teuthology.orchestra.run.smithi086.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:05:55.651 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:05:55.651 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:05:55.651 INFO:teuthology.orchestra.run.smithi086.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:05:55.652 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:05:55.652 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:05:55.652 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:05:55.652 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:05:55.652 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:05:55.652 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:05:55.652 INFO:teuthology.orchestra.run.smithi086.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:05:55.652 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:05:55.652 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:05:55.652 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:05:55.652 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:05:55.652 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:05:55.652 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:05:55.652 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:05:55.653 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:05:55.653 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:05:55.653 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:05:55.653 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:05:55.653 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:05:55.653 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:05:55.653 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:05:55.653 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:05:55.653 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:05:55.653 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:05:55.653 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:05:55.653 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:05:55.653 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:05:55.653 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:05:55.654 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:05:55.654 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:05:55.654 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:05:55.654 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:05:55.654 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:05:55.654 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:05:55.654 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:05:55.654 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:05:55.654 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:05:55.654 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:05:55.654 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:05:55.654 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:05:55.654 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:05:55.654 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:05:55.655 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:05:55.655 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:05:55.655 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:05:55.655 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:05:55.655 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:05:55.655 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:05:55.655 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:05:55.655 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:05:55.655 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:05:55.655 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:05:55.655 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:05:55.655 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:05:55.655 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:05:55.655 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:05:55.655 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:05:55.656 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:05:55.656 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:05:55.656 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:05:55.656 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:05:55.656 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:05:55.656 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:05:55.656 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:05:55.656 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:05:55.656 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:05:55.656 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:05:55.656 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:05:55.656 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:05:55.656 INFO:teuthology.orchestra.run.smithi086.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:05:55.656 INFO:teuthology.orchestra.run.smithi086.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:05:55.656 INFO:teuthology.orchestra.run.smithi086.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:05:55.657 INFO:teuthology.orchestra.run.smithi086.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:05:55.657 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:05:55.657 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:05:55.657 INFO:teuthology.orchestra.run.smithi086.stdout:Installing weak dependencies: 2024-04-26T21:05:55.657 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:05:55.657 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:05:55.657 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-04-26T21:05:55.657 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-04-26T21:05:55.657 INFO:teuthology.orchestra.run.smithi086.stdout:Install 148 Packages 2024-04-26T21:05:55.657 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrade 4 Packages 2024-04-26T21:05:55.657 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:05:55.658 INFO:teuthology.orchestra.run.smithi026.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 2.0 MB/s | 889 kB 00:00 2024-04-26T21:05:55.658 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 222 M 2024-04-26T21:05:55.658 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-04-26T21:05:55.756 INFO:teuthology.orchestra.run.smithi026.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 1.4 MB/s | 145 kB 00:00 2024-04-26T21:05:56.365 INFO:teuthology.orchestra.run.smithi026.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 3.9 MB/s | 2.4 MB 00:00 2024-04-26T21:05:56.543 INFO:teuthology.orchestra.run.smithi086.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-26T21:05:56.574 INFO:teuthology.orchestra.run.smithi026.stdout:(5/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 3.2 MB/s | 5.4 MB 00:01 2024-04-26T21:05:56.895 INFO:teuthology.orchestra.run.smithi026.stdout:(6/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 11 MB/s | 21 MB 00:02 2024-04-26T21:05:56.957 INFO:teuthology.orchestra.run.smithi026.stdout:(7/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 2.6 MB/s | 1.5 MB 00:00 2024-04-26T21:05:57.243 INFO:teuthology.orchestra.run.smithi086.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.2 MB/s | 889 kB 00:00 2024-04-26T21:05:57.614 INFO:teuthology.orchestra.run.smithi086.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 877 kB/s | 145 kB 00:00 2024-04-26T21:05:57.614 INFO:teuthology.orchestra.run.smithi026.stdout:(8/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 5.9 MB/s | 4.7 MB 00:00 2024-04-26T21:05:57.614 INFO:teuthology.orchestra.run.smithi026.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 255 kB/s | 25 kB 00:00 2024-04-26T21:05:58.166 INFO:teuthology.orchestra.run.smithi026.stdout:(10/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 13 MB/s | 17 MB 00:01 2024-04-26T21:05:58.185 INFO:teuthology.orchestra.run.smithi086.stdout:(4/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 2.6 MB/s | 5.4 MB 00:02 2024-04-26T21:05:58.241 INFO:teuthology.orchestra.run.smithi026.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 3.0 MB/s | 225 kB 00:00 2024-04-26T21:05:58.300 INFO:teuthology.orchestra.run.smithi026.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 472 kB/s | 27 kB 00:00 2024-04-26T21:05:58.359 INFO:teuthology.orchestra.run.smithi026.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 551 kB/s | 32 kB 00:00 2024-04-26T21:05:58.460 INFO:teuthology.orchestra.run.smithi086.stdout:(5/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 2.2 MB/s | 2.4 MB 00:01 2024-04-26T21:05:58.492 INFO:teuthology.orchestra.run.smithi026.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 5.5 MB/s | 747 kB 00:00 2024-04-26T21:05:58.561 INFO:teuthology.orchestra.run.smithi086.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 4.0 MB/s | 1.5 MB 00:00 2024-04-26T21:05:58.567 INFO:teuthology.orchestra.run.smithi026.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 2.2 MB/s | 163 kB 00:00 2024-04-26T21:05:58.643 INFO:teuthology.orchestra.run.smithi026.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.7 MB/s | 127 kB 00:00 2024-04-26T21:05:58.743 INFO:teuthology.orchestra.run.smithi026.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 4.9 MB/s | 501 kB 00:00 2024-04-26T21:05:59.168 INFO:teuthology.orchestra.run.smithi026.stdout:(18/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 4.8 MB/s | 11 MB 00:02 2024-04-26T21:05:59.244 INFO:teuthology.orchestra.run.smithi026.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 610 kB/s | 45 kB 00:00 2024-04-26T21:05:59.369 INFO:teuthology.orchestra.run.smithi026.stdout:(20/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 8.6 MB/s | 5.4 MB 00:00 2024-04-26T21:05:59.402 INFO:teuthology.orchestra.run.smithi026.stdout:(21/152): python3-ceph-common-19.0.0-3306.gbc61 857 kB/s | 135 kB 00:00 2024-04-26T21:05:59.436 INFO:teuthology.orchestra.run.smithi026.stdout:(22/152): python3-cephfs-19.0.0-3306.gbc612ffb. 2.4 MB/s | 161 kB 00:00 2024-04-26T21:05:59.486 INFO:teuthology.orchestra.run.smithi026.stdout:(23/152): python3-rados-19.0.0-3306.gbc612ffb.e 3.8 MB/s | 321 kB 00:00 2024-04-26T21:05:59.520 INFO:teuthology.orchestra.run.smithi026.stdout:(24/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 3.5 MB/s | 298 kB 00:00 2024-04-26T21:05:59.570 INFO:teuthology.orchestra.run.smithi026.stdout:(25/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.2 MB/s | 99 kB 00:00 2024-04-26T21:05:59.612 INFO:teuthology.orchestra.run.smithi026.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 932 kB/s | 85 kB 00:00 2024-04-26T21:05:59.619 INFO:teuthology.orchestra.run.smithi086.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 4.1 MB/s | 4.7 MB 00:01 2024-04-26T21:05:59.671 INFO:teuthology.orchestra.run.smithi026.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.9 MB/s | 171 kB 00:00 2024-04-26T21:05:59.721 INFO:teuthology.orchestra.run.smithi026.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 698 kB/s | 34 kB 00:00 2024-04-26T21:05:59.780 INFO:teuthology.orchestra.run.smithi026.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 2.4 MB/s | 142 kB 00:00 2024-04-26T21:06:00.138 INFO:teuthology.orchestra.run.smithi026.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 5.5 MB/s | 3.1 MB 00:00 2024-04-26T21:06:00.222 INFO:teuthology.orchestra.run.smithi026.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 8.0 MB/s | 3.5 MB 00:00 2024-04-26T21:06:00.289 INFO:teuthology.orchestra.run.smithi026.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 3.7 MB/s | 249 kB 00:00 2024-04-26T21:06:00.347 INFO:teuthology.orchestra.run.smithi026.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 860 kB/s | 49 kB 00:00 2024-04-26T21:06:00.406 INFO:teuthology.orchestra.run.smithi026.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 287 kB/s | 16 kB 00:00 2024-04-26T21:06:00.481 INFO:teuthology.orchestra.run.smithi026.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 3.6 MB/s | 273 kB 00:00 2024-04-26T21:06:00.615 INFO:teuthology.orchestra.run.smithi026.stdout:(36/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 5.6 MB/s | 762 kB 00:00 2024-04-26T21:06:00.682 INFO:teuthology.orchestra.run.smithi026.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 277 kB/s | 18 kB 00:00 2024-04-26T21:06:00.849 INFO:teuthology.orchestra.run.smithi026.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 485 kB/s | 80 kB 00:00 2024-04-26T21:06:00.974 INFO:teuthology.orchestra.run.smithi026.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-04-26T21:06:01.024 INFO:teuthology.orchestra.run.smithi026.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 195 kB 00:00 2024-04-26T21:06:01.074 INFO:teuthology.orchestra.run.smithi026.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-26T21:06:01.208 INFO:teuthology.orchestra.run.smithi026.stdout:(42/152): ceph-mgr-diskprediction-local-19.0.0- 6.9 MB/s | 7.4 MB 00:01 2024-04-26T21:06:01.292 INFO:teuthology.orchestra.run.smithi026.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 5.7 MB/s | 1.2 MB 00:00 2024-04-26T21:06:01.342 INFO:teuthology.orchestra.run.smithi026.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-04-26T21:06:01.367 INFO:teuthology.orchestra.run.smithi026.stdout:(45/152): python3-ply-3.11-14.el9.noarch.rpm 674 kB/s | 106 kB 00:00 2024-04-26T21:06:01.401 INFO:teuthology.orchestra.run.smithi026.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-26T21:06:01.477 INFO:teuthology.orchestra.run.smithi026.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-26T21:06:01.643 INFO:teuthology.orchestra.run.smithi026.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.8 MB/s | 689 kB 00:00 2024-04-26T21:06:01.694 INFO:teuthology.orchestra.run.smithi026.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-26T21:06:01.744 INFO:teuthology.orchestra.run.smithi026.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 3.5 MB/s | 943 kB 00:00 2024-04-26T21:06:01.769 INFO:teuthology.orchestra.run.smithi026.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 289 kB/s | 21 kB 00:00 2024-04-26T21:06:01.806 INFO:teuthology.orchestra.run.smithi026.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-04-26T21:06:01.819 INFO:teuthology.orchestra.run.smithi086.stdout:(8/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el9 4.8 MB/s | 11 MB 00:02 2024-04-26T21:06:01.836 INFO:teuthology.orchestra.run.smithi026.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 495 kB/s | 33 kB 00:00 2024-04-26T21:06:01.886 INFO:teuthology.orchestra.run.smithi086.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 382 kB/s | 25 kB 00:00 2024-04-26T21:06:01.895 INFO:teuthology.orchestra.run.smithi026.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 317 kB/s | 18 kB 00:00 2024-04-26T21:06:01.945 INFO:teuthology.orchestra.run.smithi026.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 227 kB/s | 11 kB 00:00 2024-04-26T21:06:02.029 INFO:teuthology.orchestra.run.smithi026.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-04-26T21:06:02.063 INFO:teuthology.orchestra.run.smithi026.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-04-26T21:06:02.088 INFO:teuthology.orchestra.run.smithi026.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 785 kB/s | 45 kB 00:00 2024-04-26T21:06:02.120 INFO:teuthology.orchestra.run.smithi086.stdout:(10/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 4.7 MB/s | 17 MB 00:03 2024-04-26T21:06:02.138 INFO:teuthology.orchestra.run.smithi026.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 662 kB 00:00 2024-04-26T21:06:02.180 INFO:teuthology.orchestra.run.smithi026.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 6.3 MB/s | 582 kB 00:00 2024-04-26T21:06:02.195 INFO:teuthology.orchestra.run.smithi086.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 3.0 MB/s | 225 kB 00:00 2024-04-26T21:06:02.205 INFO:teuthology.orchestra.run.smithi026.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 382 kB/s | 25 kB 00:00 2024-04-26T21:06:02.231 INFO:teuthology.orchestra.run.smithi026.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 961 kB/s | 47 kB 00:00 2024-04-26T21:06:02.267 INFO:teuthology.orchestra.run.smithi086.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 400 kB/s | 27 kB 00:00 2024-04-26T21:06:02.273 INFO:teuthology.orchestra.run.smithi026.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-04-26T21:06:02.306 INFO:teuthology.orchestra.run.smithi026.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-04-26T21:06:02.321 INFO:teuthology.orchestra.run.smithi086.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 569 kB/s | 32 kB 00:00 2024-04-26T21:06:02.360 INFO:teuthology.orchestra.run.smithi026.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 407 kB/s | 35 kB 00:00 2024-04-26T21:06:02.455 INFO:teuthology.orchestra.run.smithi086.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 5.5 MB/s | 747 kB 00:00 2024-04-26T21:06:02.514 INFO:teuthology.orchestra.run.smithi086.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 2.8 MB/s | 163 kB 00:00 2024-04-26T21:06:02.532 INFO:teuthology.orchestra.run.smithi026.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2024-04-26T21:06:02.597 INFO:teuthology.orchestra.run.smithi086.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.5 MB/s | 127 kB 00:00 2024-04-26T21:06:02.640 INFO:teuthology.orchestra.run.smithi026.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-26T21:06:02.697 INFO:teuthology.orchestra.run.smithi086.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 4.9 MB/s | 501 kB 00:00 2024-04-26T21:06:02.724 INFO:teuthology.orchestra.run.smithi026.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-04-26T21:06:02.766 INFO:teuthology.orchestra.run.smithi026.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 7.2 MB/s | 911 kB 00:00 2024-04-26T21:06:02.799 INFO:teuthology.orchestra.run.smithi026.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.8 MB/s | 133 kB 00:00 2024-04-26T21:06:02.892 INFO:teuthology.orchestra.run.smithi026.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-26T21:06:02.942 INFO:teuthology.orchestra.run.smithi026.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-26T21:06:03.059 INFO:teuthology.orchestra.run.smithi026.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-04-26T21:06:03.134 INFO:teuthology.orchestra.run.smithi026.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-04-26T21:06:03.268 INFO:teuthology.orchestra.run.smithi026.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-04-26T21:06:03.326 INFO:teuthology.orchestra.run.smithi026.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.3 MB/s | 248 kB 00:00 2024-04-26T21:06:03.343 INFO:teuthology.orchestra.run.smithi026.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 345 kB/s | 26 kB 00:00 2024-04-26T21:06:03.364 INFO:teuthology.orchestra.run.smithi086.stdout:(18/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 8.1 MB/s | 5.4 MB 00:00 2024-04-26T21:06:03.368 INFO:teuthology.orchestra.run.smithi026.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 465 kB/s | 19 kB 00:00 2024-04-26T21:06:03.402 INFO:teuthology.orchestra.run.smithi026.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-26T21:06:03.414 INFO:teuthology.orchestra.run.smithi086.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 923 kB/s | 45 kB 00:00 2024-04-26T21:06:03.419 INFO:teuthology.orchestra.run.smithi026.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 705 kB/s | 35 kB 00:00 2024-04-26T21:06:03.494 INFO:teuthology.orchestra.run.smithi026.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-04-26T21:06:03.506 INFO:teuthology.orchestra.run.smithi086.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.5 MB/s | 135 kB 00:00 2024-04-26T21:06:03.553 INFO:teuthology.orchestra.run.smithi026.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.8 MB/s | 222 kB 00:00 2024-04-26T21:06:03.565 INFO:teuthology.orchestra.run.smithi086.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 2.7 MB/s | 161 kB 00:00 2024-04-26T21:06:03.603 INFO:teuthology.orchestra.run.smithi026.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-26T21:06:03.650 INFO:teuthology.orchestra.run.smithi086.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 3.8 MB/s | 321 kB 00:00 2024-04-26T21:06:03.704 INFO:teuthology.orchestra.run.smithi026.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-04-26T21:06:03.723 INFO:teuthology.orchestra.run.smithi086.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 3.9 MB/s | 298 kB 00:00 2024-04-26T21:06:03.729 INFO:teuthology.orchestra.run.smithi026.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 340 kB/s | 42 kB 00:00 2024-04-26T21:06:03.771 INFO:teuthology.orchestra.run.smithi026.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 269 kB 00:00 2024-04-26T21:06:03.804 INFO:teuthology.orchestra.run.smithi026.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-26T21:06:03.805 INFO:teuthology.orchestra.run.smithi086.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.2 MB/s | 99 kB 00:00 2024-04-26T21:06:03.855 INFO:teuthology.orchestra.run.smithi026.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-04-26T21:06:03.874 INFO:teuthology.orchestra.run.smithi086.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.2 MB/s | 85 kB 00:00 2024-04-26T21:06:03.880 INFO:teuthology.orchestra.run.smithi026.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 685 kB/s | 51 kB 00:00 2024-04-26T21:06:03.913 INFO:teuthology.orchestra.run.smithi026.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 927 kB/s | 54 kB 00:00 2024-04-26T21:06:03.980 INFO:teuthology.orchestra.run.smithi026.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 634 kB/s | 42 kB 00:00 2024-04-26T21:06:04.031 INFO:teuthology.orchestra.run.smithi026.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 900 kB/s | 44 kB 00:00 2024-04-26T21:06:04.089 INFO:teuthology.orchestra.run.smithi026.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.8 MB/s | 221 kB 00:00 2024-04-26T21:06:04.140 INFO:teuthology.orchestra.run.smithi026.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 733 kB/s | 36 kB 00:00 2024-04-26T21:06:04.215 INFO:teuthology.orchestra.run.smithi026.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 640 kB/s | 47 kB 00:00 2024-04-26T21:06:04.274 INFO:teuthology.orchestra.run.smithi026.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-04-26T21:06:04.274 INFO:teuthology.orchestra.run.smithi086.stdout:(26/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 7.7 MB/s | 3.1 MB 00:00 2024-04-26T21:06:04.324 INFO:teuthology.orchestra.run.smithi026.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 1.2 MB/s | 58 kB 00:00 2024-04-26T21:06:04.350 INFO:teuthology.orchestra.run.smithi086.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.2 MB/s | 171 kB 00:00 2024-04-26T21:06:04.374 INFO:teuthology.orchestra.run.smithi026.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.9 MB/s | 94 kB 00:00 2024-04-26T21:06:04.418 INFO:teuthology.orchestra.run.smithi086.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 507 kB/s | 34 kB 00:00 2024-04-26T21:06:04.424 INFO:teuthology.orchestra.run.smithi026.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-04-26T21:06:04.474 INFO:teuthology.orchestra.run.smithi026.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 915 kB/s | 45 kB 00:00 2024-04-26T21:06:04.484 INFO:teuthology.orchestra.run.smithi086.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 2.2 MB/s | 142 kB 00:00 2024-04-26T21:06:04.708 INFO:teuthology.orchestra.run.smithi026.stdout:(101/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-04-26T21:06:04.842 INFO:teuthology.orchestra.run.smithi026.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 61 kB/s | 22 kB 00:00 2024-04-26T21:06:04.909 INFO:teuthology.orchestra.run.smithi086.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 8.3 MB/s | 3.5 MB 00:00 2024-04-26T21:06:05.067 INFO:teuthology.orchestra.run.smithi026.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-04-26T21:06:05.409 INFO:teuthology.orchestra.run.smithi026.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 907 kB/s | 309 kB 00:00 2024-04-26T21:06:05.443 INFO:teuthology.orchestra.run.smithi026.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 915 kB/s | 548 kB 00:00 2024-04-26T21:06:05.485 INFO:teuthology.orchestra.run.smithi026.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 270 kB/s | 19 kB 00:00 2024-04-26T21:06:05.652 INFO:teuthology.orchestra.run.smithi026.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 156 kB/s | 26 kB 00:00 2024-04-26T21:06:05.726 INFO:teuthology.orchestra.run.smithi086.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 9.1 MB/s | 7.4 MB 00:00 2024-04-26T21:06:05.744 INFO:teuthology.orchestra.run.smithi026.stdout:(108/152): libarrow-9.0.0-11.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-04-26T21:06:05.760 INFO:teuthology.orchestra.run.smithi026.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 468 kB/s | 49 kB 00:00 2024-04-26T21:06:05.819 INFO:teuthology.orchestra.run.smithi026.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 913 kB/s | 67 kB 00:00 2024-04-26T21:06:05.844 INFO:teuthology.orchestra.run.smithi026.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-26T21:06:05.911 INFO:teuthology.orchestra.run.smithi026.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.2 MB/s | 839 kB 00:00 2024-04-26T21:06:05.953 INFO:teuthology.orchestra.run.smithi026.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 5.0 MB/s | 548 kB 00:00 2024-04-26T21:06:05.987 INFO:teuthology.orchestra.run.smithi026.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 628 kB/s | 43 kB 00:00 2024-04-26T21:06:06.001 INFO:teuthology.orchestra.run.smithi086.stdout:(32/152): ceph-common-19.0.0-3306.gbc612ffb.el9 2.1 MB/s | 21 MB 00:09 2024-04-26T21:06:06.029 INFO:teuthology.orchestra.run.smithi026.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 429 kB/s | 32 kB 00:00 2024-04-26T21:06:06.035 INFO:teuthology.orchestra.run.smithi086.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3306.gbc 809 kB/s | 249 kB 00:00 2024-04-26T21:06:06.068 INFO:teuthology.orchestra.run.smithi086.stdout:(34/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 752 kB/s | 49 kB 00:00 2024-04-26T21:06:06.082 INFO:teuthology.orchestra.run.smithi026.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 164 kB/s | 15 kB 00:00 2024-04-26T21:06:06.094 INFO:teuthology.orchestra.run.smithi086.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3306.gb 284 kB/s | 16 kB 00:00 2024-04-26T21:06:06.113 INFO:teuthology.orchestra.run.smithi026.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-04-26T21:06:06.164 INFO:teuthology.orchestra.run.smithi026.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.3 MB/s | 358 kB 00:00 2024-04-26T21:06:06.197 INFO:teuthology.orchestra.run.smithi026.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.6 MB/s | 303 kB 00:00 2024-04-26T21:06:06.210 INFO:teuthology.orchestra.run.smithi086.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 1.9 MB/s | 273 kB 00:00 2024-04-26T21:06:06.244 INFO:teuthology.orchestra.run.smithi086.stdout:(37/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 5.0 MB/s | 762 kB 00:00 2024-04-26T21:06:06.310 INFO:teuthology.orchestra.run.smithi086.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 188 kB/s | 18 kB 00:00 2024-04-26T21:06:06.698 INFO:teuthology.orchestra.run.smithi026.stdout:(120/152): ceph-test-19.0.0-3306.gbc612ffb.el9. 5.3 MB/s | 49 MB 00:09 2024-04-26T21:06:06.723 INFO:teuthology.orchestra.run.smithi026.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 64 kB/s | 35 kB 00:00 2024-04-26T21:06:06.748 INFO:teuthology.orchestra.run.smithi026.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 374 kB/s | 206 kB 00:00 2024-04-26T21:06:06.831 INFO:teuthology.orchestra.run.smithi026.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 16 MB/s | 2.0 MB 00:00 2024-04-26T21:06:06.857 INFO:teuthology.orchestra.run.smithi026.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-26T21:06:06.882 INFO:teuthology.orchestra.run.smithi026.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 90 kB/s | 11 kB 00:00 2024-04-26T21:06:06.907 INFO:teuthology.orchestra.run.smithi026.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 242 kB/s | 18 kB 00:00 2024-04-26T21:06:06.970 INFO:teuthology.orchestra.run.smithi086.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 111 kB/s | 80 kB 00:00 2024-04-26T21:06:06.982 INFO:teuthology.orchestra.run.smithi026.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 234 kB/s | 23 kB 00:00 2024-04-26T21:06:07.007 INFO:teuthology.orchestra.run.smithi026.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 157 kB/s | 19 kB 00:00 2024-04-26T21:06:07.033 INFO:teuthology.orchestra.run.smithi026.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 157 kB/s | 20 kB 00:00 2024-04-26T21:06:07.108 INFO:teuthology.orchestra.run.smithi026.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-04-26T21:06:07.133 INFO:teuthology.orchestra.run.smithi026.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 373 kB/s | 46 kB 00:00 2024-04-26T21:06:07.159 INFO:teuthology.orchestra.run.smithi026.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 630 kB/s | 79 kB 00:00 2024-04-26T21:06:07.209 INFO:teuthology.orchestra.run.smithi026.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 788 kB/s | 58 kB 00:00 2024-04-26T21:06:07.242 INFO:teuthology.orchestra.run.smithi026.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-04-26T21:06:07.268 INFO:teuthology.orchestra.run.smithi026.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 152 kB/s | 16 kB 00:00 2024-04-26T21:06:07.293 INFO:teuthology.orchestra.run.smithi026.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-04-26T21:06:07.343 INFO:teuthology.orchestra.run.smithi026.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 385 kB/s | 28 kB 00:00 2024-04-26T21:06:07.368 INFO:teuthology.orchestra.run.smithi026.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 313 kB/s | 31 kB 00:00 2024-04-26T21:06:07.402 INFO:teuthology.orchestra.run.smithi026.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-26T21:06:07.452 INFO:teuthology.orchestra.run.smithi026.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 734 kB/s | 59 kB 00:00 2024-04-26T21:06:07.477 INFO:teuthology.orchestra.run.smithi026.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 339 kB/s | 36 kB 00:00 2024-04-26T21:06:07.503 INFO:teuthology.orchestra.run.smithi026.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 510 kB/s | 51 kB 00:00 2024-04-26T21:06:07.553 INFO:teuthology.orchestra.run.smithi026.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.1 MB/s | 230 kB 00:00 2024-04-26T21:06:07.578 INFO:teuthology.orchestra.run.smithi026.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 913 kB/s | 90 kB 00:00 2024-04-26T21:06:07.612 INFO:teuthology.orchestra.run.smithi026.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.9 MB/s | 427 kB 00:00 2024-04-26T21:06:07.654 INFO:teuthology.orchestra.run.smithi026.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 285 kB/s | 20 kB 00:00 2024-04-26T21:06:07.679 INFO:teuthology.orchestra.run.smithi026.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-04-26T21:06:07.746 INFO:teuthology.orchestra.run.smithi026.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-26T21:06:07.762 INFO:teuthology.orchestra.run.smithi086.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 247 kB/s | 195 kB 00:00 2024-04-26T21:06:07.887 INFO:teuthology.orchestra.run.smithi086.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 509 kB/s | 803 kB 00:01 2024-04-26T21:06:07.946 INFO:teuthology.orchestra.run.smithi026.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 6.8 MB/s | 1.3 MB 00:00 2024-04-26T21:06:08.071 INFO:teuthology.orchestra.run.smithi026.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-04-26T21:06:08.221 INFO:teuthology.orchestra.run.smithi026.stdout:(151/152): librados2-19.0.0-3306.gbc612ffb.el9. 6.4 MB/s | 3.4 MB 00:00 2024-04-26T21:06:08.396 INFO:teuthology.orchestra.run.smithi026.stdout:(152/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 4.5 MB/s | 3.1 MB 00:00 2024-04-26T21:06:08.404 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:06:08.405 INFO:teuthology.orchestra.run.smithi026.stdout:Total 16 MB/s | 222 MB 00:14 2024-04-26T21:06:08.687 INFO:teuthology.orchestra.run.smithi086.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 274 kB/s | 253 kB 00:00 2024-04-26T21:06:09.686 INFO:teuthology.orchestra.run.smithi086.stdout:(43/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 6.3 MB/s | 49 MB 00:07 2024-04-26T21:06:10.021 INFO:teuthology.orchestra.run.smithi086.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 80 kB/s | 106 kB 00:01 2024-04-26T21:06:10.079 INFO:teuthology.orchestra.run.smithi086.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 573 kB/s | 1.2 MB 00:02 2024-04-26T21:06:10.472 INFO:teuthology.orchestra.run.smithi086.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 172 kB/s | 135 kB 00:00 2024-04-26T21:06:10.663 INFO:teuthology.orchestra.run.smithi086.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-04-26T21:06:10.755 INFO:teuthology.orchestra.run.smithi086.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 10 MB/s | 943 kB 00:00 2024-04-26T21:06:10.789 INFO:teuthology.orchestra.run.smithi086.stdout:(49/152): python3-requests-2.25.1-8.el9.noarch. 163 kB/s | 125 kB 00:00 2024-04-26T21:06:10.814 INFO:teuthology.orchestra.run.smithi086.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-26T21:06:10.864 INFO:teuthology.orchestra.run.smithi086.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 6.2 MB/s | 313 kB 00:00 2024-04-26T21:06:10.912 INFO:teuthology.orchestra.run.smithi086.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 176 kB/s | 21 kB 00:00 2024-04-26T21:06:10.948 INFO:teuthology.orchestra.run.smithi086.stdout:(53/152): python3-urllib3-1.26.5-5.el9.noarch.r 247 kB/s | 215 kB 00:00 2024-04-26T21:06:10.973 INFO:teuthology.orchestra.run.smithi086.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 303 kB/s | 33 kB 00:00 2024-04-26T21:06:11.024 INFO:teuthology.orchestra.run.smithi086.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 378 kB/s | 18 kB 00:00 2024-04-26T21:06:11.058 INFO:teuthology.orchestra.run.smithi086.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 136 kB/s | 11 kB 00:00 2024-04-26T21:06:11.091 INFO:teuthology.orchestra.run.smithi086.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-04-26T21:06:11.125 INFO:teuthology.orchestra.run.smithi086.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 685 kB/s | 45 kB 00:00 2024-04-26T21:06:11.183 INFO:teuthology.orchestra.run.smithi086.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-04-26T21:06:11.251 INFO:teuthology.orchestra.run.smithi086.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 385 kB/s | 25 kB 00:00 2024-04-26T21:06:11.318 INFO:teuthology.orchestra.run.smithi086.stdout:(61/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.4 MB/s | 3.0 MB 00:00 2024-04-26T21:06:11.342 INFO:teuthology.orchestra.run.smithi086.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 522 kB/s | 47 kB 00:00 2024-04-26T21:06:11.376 INFO:teuthology.orchestra.run.smithi086.stdout:(63/152): libtool-2.4.6-45.el9.x86_64.rpm 2.3 MB/s | 582 kB 00:00 2024-04-26T21:06:11.396 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-04-26T21:06:11.410 INFO:teuthology.orchestra.run.smithi086.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-04-26T21:06:11.435 INFO:teuthology.orchestra.run.smithi086.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 612 kB/s | 35 kB 00:00 2024-04-26T21:06:11.460 INFO:teuthology.orchestra.run.smithi086.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-04-26T21:06:11.578 INFO:teuthology.orchestra.run.smithi086.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-26T21:06:11.662 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-04-26T21:06:11.663 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-04-26T21:06:11.694 INFO:teuthology.orchestra.run.smithi086.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-26T21:06:11.786 INFO:teuthology.orchestra.run.smithi086.stdout:(69/152): openblas-openmp-0.3.21-2.el9.x86_64.r 13 MB/s | 4.7 MB 00:00 2024-04-26T21:06:11.845 INFO:teuthology.orchestra.run.smithi086.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 890 kB/s | 133 kB 00:00 2024-04-26T21:06:11.895 INFO:teuthology.orchestra.run.smithi086.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 2.8 MB/s | 911 kB 00:00 2024-04-26T21:06:11.954 INFO:teuthology.orchestra.run.smithi086.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 9.4 MB/s | 1.0 MB 00:00 2024-04-26T21:06:11.979 INFO:teuthology.orchestra.run.smithi086.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-04-26T21:06:12.038 INFO:teuthology.orchestra.run.smithi086.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-26T21:06:12.088 INFO:teuthology.orchestra.run.smithi086.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-26T21:06:12.146 INFO:teuthology.orchestra.run.smithi086.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 445 kB/s | 26 kB 00:00 2024-04-26T21:06:12.197 INFO:teuthology.orchestra.run.smithi086.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 390 kB/s | 19 kB 00:00 2024-04-26T21:06:12.322 INFO:teuthology.orchestra.run.smithi086.stdout:(78/152): postgresql-server-13.14-1.el9.x86_64. 11 MB/s | 5.8 MB 00:00 2024-04-26T21:06:12.356 INFO:teuthology.orchestra.run.smithi086.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-04-26T21:06:12.381 INFO:teuthology.orchestra.run.smithi086.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 603 kB/s | 35 kB 00:00 2024-04-26T21:06:12.523 INFO:teuthology.orchestra.run.smithi086.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-04-26T21:06:12.556 INFO:teuthology.orchestra.run.smithi086.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 250 kB 00:00 2024-04-26T21:06:12.590 INFO:teuthology.orchestra.run.smithi086.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-26T21:06:12.615 INFO:teuthology.orchestra.run.smithi086.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-26T21:06:12.640 INFO:teuthology.orchestra.run.smithi086.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 855 kB/s | 42 kB 00:00 2024-04-26T21:06:12.683 INFO:teuthology.orchestra.run.smithi086.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 269 kB 00:00 2024-04-26T21:06:12.746 INFO:teuthology.orchestra.run.smithi086.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-04-26T21:06:12.783 INFO:teuthology.orchestra.run.smithi086.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-04-26T21:06:12.808 INFO:teuthology.orchestra.run.smithi086.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 826 kB/s | 51 kB 00:00 2024-04-26T21:06:12.834 INFO:teuthology.orchestra.run.smithi086.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-26T21:06:12.892 INFO:teuthology.orchestra.run.smithi086.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 723 kB/s | 42 kB 00:00 2024-04-26T21:06:13.018 INFO:teuthology.orchestra.run.smithi086.stdout:(92/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.7 MB/s | 5.1 MB 00:00 2024-04-26T21:06:13.068 INFO:teuthology.orchestra.run.smithi086.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 253 kB/s | 44 kB 00:00 2024-04-26T21:06:13.101 INFO:teuthology.orchestra.run.smithi086.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-04-26T21:06:13.135 INFO:teuthology.orchestra.run.smithi086.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 544 kB/s | 36 kB 00:00 2024-04-26T21:06:13.161 INFO:teuthology.orchestra.run.smithi086.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 811 kB/s | 47 kB 00:00 2024-04-26T21:06:13.194 INFO:teuthology.orchestra.run.smithi086.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-26T21:06:13.220 INFO:teuthology.orchestra.run.smithi086.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-04-26T21:06:13.270 INFO:teuthology.orchestra.run.smithi086.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-04-26T21:06:13.320 INFO:teuthology.orchestra.run.smithi086.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-04-26T21:06:13.379 INFO:teuthology.orchestra.run.smithi086.stdout:(101/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 511 kB/s | 94 kB 00:00 2024-04-26T21:06:13.429 INFO:teuthology.orchestra.run.smithi086.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 208 kB/s | 22 kB 00:00 2024-04-26T21:06:13.496 INFO:teuthology.orchestra.run.smithi086.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 7.3 MB/s | 863 kB 00:00 2024-04-26T21:06:13.822 INFO:teuthology.orchestra.run.smithi086.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:01 2024-04-26T21:06:14.097 INFO:teuthology.orchestra.run.smithi086.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 71 kB/s | 19 kB 00:00 2024-04-26T21:06:14.147 INFO:teuthology.orchestra.run.smithi086.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 765 kB/s | 548 kB 00:00 2024-04-26T21:06:14.181 INFO:teuthology.orchestra.run.smithi086.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 452 kB/s | 309 kB 00:00 2024-04-26T21:06:14.268 INFO:teuthology.orchestra.run.smithi086.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 216 kB/s | 26 kB 00:00 2024-04-26T21:06:14.289 INFO:teuthology.orchestra.run.smithi086.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 450 kB/s | 49 kB 00:00 2024-04-26T21:06:14.373 INFO:teuthology.orchestra.run.smithi086.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 650 kB/s | 67 kB 00:00 2024-04-26T21:06:14.423 INFO:teuthology.orchestra.run.smithi086.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-04-26T21:06:14.510 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-04-26T21:06:14.513 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-04-26T21:06:14.590 INFO:teuthology.orchestra.run.smithi086.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.8 MB/s | 839 kB 00:00 2024-04-26T21:06:14.648 INFO:teuthology.orchestra.run.smithi086.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 2.4 MB/s | 548 kB 00:00 2024-04-26T21:06:14.690 INFO:teuthology.orchestra.run.smithi086.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 437 kB/s | 43 kB 00:00 2024-04-26T21:06:14.758 INFO:teuthology.orchestra.run.smithi086.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 296 kB/s | 32 kB 00:00 2024-04-26T21:06:14.791 INFO:teuthology.orchestra.run.smithi086.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-04-26T21:06:14.933 INFO:teuthology.orchestra.run.smithi086.stdout:(117/152): libarrow-9.0.0-11.el9.x86_64.rpm 5.3 MB/s | 4.4 MB 00:00 2024-04-26T21:06:14.967 INFO:teuthology.orchestra.run.smithi086.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 825 kB/s | 172 kB 00:00 2024-04-26T21:06:15.000 INFO:teuthology.orchestra.run.smithi086.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 1.7 MB/s | 358 kB 00:00 2024-04-26T21:06:15.042 INFO:teuthology.orchestra.run.smithi086.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-26T21:06:15.068 INFO:teuthology.orchestra.run.smithi086.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 353 kB/s | 35 kB 00:00 2024-04-26T21:06:15.110 INFO:teuthology.orchestra.run.smithi086.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.9 MB/s | 206 kB 00:00 2024-04-26T21:06:15.218 INFO:teuthology.orchestra.run.smithi086.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-26T21:06:15.243 INFO:teuthology.orchestra.run.smithi086.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 826 kB/s | 144 kB 00:00 2024-04-26T21:06:15.268 INFO:teuthology.orchestra.run.smithi086.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 67 kB/s | 11 kB 00:00 2024-04-26T21:06:15.319 INFO:teuthology.orchestra.run.smithi086.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 177 kB/s | 18 kB 00:00 2024-04-26T21:06:15.377 INFO:teuthology.orchestra.run.smithi086.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 181 kB/s | 19 kB 00:00 2024-04-26T21:06:15.403 INFO:teuthology.orchestra.run.smithi086.stdout:(128/152): python3-jaraco-collections-3.0.0-8.e 174 kB/s | 23 kB 00:00 2024-04-26T21:06:15.428 INFO:teuthology.orchestra.run.smithi086.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-04-26T21:06:15.511 INFO:teuthology.orchestra.run.smithi086.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 429 kB/s | 46 kB 00:00 2024-04-26T21:06:15.538 INFO:teuthology.orchestra.run.smithi086.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 729 kB/s | 79 kB 00:00 2024-04-26T21:06:15.595 INFO:teuthology.orchestra.run.smithi086.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-04-26T21:06:15.621 INFO:teuthology.orchestra.run.smithi086.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 534 kB/s | 58 kB 00:00 2024-04-26T21:06:15.654 INFO:teuthology.orchestra.run.smithi086.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-26T21:06:15.696 INFO:teuthology.orchestra.run.smithi086.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 165 kB/s | 16 kB 00:00 2024-04-26T21:06:15.721 INFO:teuthology.orchestra.run.smithi086.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 901 kB/s | 90 kB 00:00 2024-04-26T21:06:15.755 INFO:teuthology.orchestra.run.smithi086.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 282 kB/s | 28 kB 00:00 2024-04-26T21:06:15.822 INFO:teuthology.orchestra.run.smithi086.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 246 kB/s | 31 kB 00:00 2024-04-26T21:06:15.856 INFO:teuthology.orchestra.run.smithi086.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-04-26T21:06:15.881 INFO:teuthology.orchestra.run.smithi086.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 473 kB/s | 59 kB 00:00 2024-04-26T21:06:15.931 INFO:teuthology.orchestra.run.smithi086.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 330 kB/s | 36 kB 00:00 2024-04-26T21:06:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 510 kB/s | 51 kB 00:00 2024-04-26T21:06:15.991 INFO:teuthology.orchestra.run.smithi086.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-26T21:06:16.023 INFO:teuthology.orchestra.run.smithi086.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 979 kB/s | 90 kB 00:00 2024-04-26T21:06:16.093 INFO:teuthology.orchestra.run.smithi086.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-04-26T21:06:16.124 INFO:teuthology.orchestra.run.smithi086.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 149 kB/s | 20 kB 00:00 2024-04-26T21:06:16.158 INFO:teuthology.orchestra.run.smithi086.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-04-26T21:06:16.250 INFO:teuthology.orchestra.run.smithi086.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-04-26T21:06:16.659 INFO:teuthology.orchestra.run.smithi086.stdout:(149/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 6.3 MB/s | 3.1 MB 00:00 2024-04-26T21:06:16.792 INFO:teuthology.orchestra.run.smithi086.stdout:(150/152): librados2-19.0.0-3306.gbc612ffb.el9. 5.1 MB/s | 3.4 MB 00:00 2024-04-26T21:06:18.603 INFO:teuthology.orchestra.run.smithi086.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 588 kB/s | 1.3 MB 00:02 2024-04-26T21:06:20.417 INFO:teuthology.orchestra.run.smithi086.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 653 kB/s | 2.4 MB 00:03 2024-04-26T21:06:20.426 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:06:20.427 INFO:teuthology.orchestra.run.smithi086.stdout:Total 8.9 MB/s | 222 MB 00:24 2024-04-26T21:06:21.029 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-04-26T21:06:21.129 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:06:21.214 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:06:21.348 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:06:21.410 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:06:21.447 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:06:21.586 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:06:22.034 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:06:22.136 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:06:22.306 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:06:22.336 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:06:22.464 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:06:22.510 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:06:22.874 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:06:22.935 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:06:22.965 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:06:23.141 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:06:23.166 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:06:23.236 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-04-26T21:06:23.264 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:06:23.292 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:06:23.375 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:06:23.485 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-04-26T21:06:23.485 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-04-26T21:06:23.742 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:06:23.803 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:06:24.008 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:06:24.139 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:06:24.185 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:06:24.236 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:06:24.346 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:06:24.401 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:06:24.466 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:06:24.513 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:06:24.575 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:06:24.705 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:06:24.775 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:06:24.811 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:06:24.872 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:06:24.993 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:06:25.031 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:06:25.086 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:06:25.122 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:06:25.243 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:06:25.301 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:06:25.410 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:06:25.590 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:06:25.662 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:06:26.230 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-04-26T21:06:26.231 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-04-26T21:06:26.390 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:06:26.522 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:06:26.703 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:06:26.753 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:06:26.802 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:06:26.879 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:06:27.164 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:06:27.222 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:06:27.304 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:06:27.405 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:06:27.447 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:06:27.488 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:06:27.557 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:06:27.602 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:06:27.667 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:06:27.732 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:06:27.944 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:06:28.000 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:06:28.064 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:06:28.094 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:06:28.240 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:06:28.286 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:06:28.332 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:06:28.368 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:06:28.401 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:06:28.598 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:06:28.776 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:06:28.825 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:06:28.864 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:06:28.911 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:06:28.980 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:06:29.049 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:06:29.117 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:06:29.177 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:06:29.227 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:06:29.302 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:06:29.348 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:06:29.497 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:06:29.565 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:06:29.606 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:06:29.679 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:06:29.840 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:06:29.910 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:06:30.922 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:06:30.976 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:06:31.009 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:06:31.148 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:06:31.222 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:06:31.333 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:06:31.377 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:06:31.611 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:06:31.640 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:06:31.759 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:06:32.631 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:06:32.762 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-04-26T21:06:32.792 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:06:32.865 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:06:32.958 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:06:33.081 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:06:33.147 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:06:33.184 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:06:33.352 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:06:33.820 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:06:33.965 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:06:34.118 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:06:34.178 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:06:34.312 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:06:34.357 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:06:34.499 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:06:34.596 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:06:34.650 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:06:34.701 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:06:34.778 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:06:34.857 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:06:34.886 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:06:35.063 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:06:35.121 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:06:35.122 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:06:35.209 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:06:35.217 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:06:35.239 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:06:35.312 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:06:35.346 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:06:35.383 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:06:35.480 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:06:35.660 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:06:35.749 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:06:35.950 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:06:36.076 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:06:36.085 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:06:36.117 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:06:36.120 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:06:36.164 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:06:36.222 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:06:36.247 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:06:36.264 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:06:36.313 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:06:36.381 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:06:36.424 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:06:36.464 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:06:36.579 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:06:36.689 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:06:36.733 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:06:36.804 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:06:36.919 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:06:36.986 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:06:37.038 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:06:37.077 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:06:37.196 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:06:37.289 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:06:37.396 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:06:37.581 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:06:37.648 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:06:38.336 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:06:38.438 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:06:38.499 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:06:38.546 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:06:38.596 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:06:38.676 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:06:38.974 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:06:39.045 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:06:39.084 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:06:39.112 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:06:39.200 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:06:39.301 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:06:39.343 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:06:39.417 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:06:39.499 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:06:39.543 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:06:39.606 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:06:39.669 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:06:39.770 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:06:39.821 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:06:39.901 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:06:39.955 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:06:40.006 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:06:40.006 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:06:40.006 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:06:40.006 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:06:40.020 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:06:40.072 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:06:40.087 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:06:40.224 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:06:40.259 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:06:40.334 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:06:40.375 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:06:40.416 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:06:40.629 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:06:40.827 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:06:40.872 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:06:40.937 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:06:41.012 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:06:41.092 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:06:41.153 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:06:41.203 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:06:41.266 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:06:41.315 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:06:41.400 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:06:41.469 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:06:41.614 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:06:41.674 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:06:41.718 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:06:41.787 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:06:41.965 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:06:42.031 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:06:42.990 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:06:43.039 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:06:43.097 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:06:43.222 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:06:43.284 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:06:43.396 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:06:43.441 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:06:43.673 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:06:43.714 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:06:43.834 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:06:44.645 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:06:44.793 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:06:46.380 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:06:46.477 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:06:46.529 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:06:46.581 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:06:46.973 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:06:47.046 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:06:47.172 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:06:47.205 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:06:47.260 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:06:47.842 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:06:47.876 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:06:48.002 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:06:48.028 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:06:50.721 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:06:50.767 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:06:51.436 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:06:51.487 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:06:51.656 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:06:51.656 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:06:51.657 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:06:51.657 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:06:51.705 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:06:53.558 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:06:53.558 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /sys 2024-04-26T21:06:53.558 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /proc 2024-04-26T21:06:53.559 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /mnt 2024-04-26T21:06:53.559 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /var/tmp 2024-04-26T21:06:53.559 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /home 2024-04-26T21:06:53.559 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /root 2024-04-26T21:06:53.559 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /tmp 2024-04-26T21:06:53.559 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:06:53.802 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:06:53.853 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:06:53.853 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:53.853 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:06:53.853 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:06:55.546 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:06:55.629 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:06:55.629 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:55.629 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:06:55.629 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:06:55.629 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:06:55.629 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:06:55.896 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:06:55.975 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:06:55.976 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:55.976 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:06:55.976 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:06:55.976 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:06:55.976 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:06:56.488 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:06:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:06:56.562 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:06:56.562 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:06:56.562 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:06:56.562 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:06:56.562 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:06:56.706 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:06:56.770 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:06:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:06:57.090 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:06:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:06:57.259 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:06:57.376 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:06:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:06:57.609 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:06:57.658 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:06:57.700 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:06:58.371 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:06:58.412 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:06:58.472 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:06:58.587 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:06:58.629 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:06:59.976 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:07:00.014 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:07:01.178 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:07:01.206 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:07:01.360 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:07:01.545 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:07:01.575 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:07:01.654 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:07:01.654 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:07:01.654 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:07:01.654 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:07:01.655 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:07:01.655 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:01.717 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:07:01.756 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:07:01.872 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:07:01.936 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:07:03.000 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:07:03.036 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:07:03.107 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:07:03.107 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:07:03.107 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:07:03.108 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:07:03.108 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:07:03.108 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:03.153 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:07:03.227 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:07:03.227 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:07:03.227 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:07:03.227 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:03.313 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:07:03.392 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:07:03.392 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:07:03.392 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:07:03.392 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:07:03.393 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:07:03.393 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:03.763 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:07:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:07:03.844 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:07:03.844 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:07:03.844 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:07:03.844 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:07:03.844 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:05.002 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:07:05.002 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /sys 2024-04-26T21:07:05.002 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /proc 2024-04-26T21:07:05.003 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /mnt 2024-04-26T21:07:05.003 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /var/tmp 2024-04-26T21:07:05.003 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /home 2024-04-26T21:07:05.003 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /root 2024-04-26T21:07:05.003 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /tmp 2024-04-26T21:07:05.003 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:05.251 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:07:05.302 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:07:05.302 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:07:05.303 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:07:05.303 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:06.935 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:07:07.010 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:07:07.010 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:07:07.010 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:07:07.010 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:07:07.010 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:07:07.010 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:07.292 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:07:07.360 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:07:07.361 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:07:07.361 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:07:07.361 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:07:07.361 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:07:07.361 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:07.853 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:07:07.937 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:07:07.938 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:07:07.938 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:07:07.938 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:07:07.938 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:07:07.938 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:08.078 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:07:08.143 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:07:08.218 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:07:08.460 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:07:08.558 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:07:08.659 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:07:08.778 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:07:08.943 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:07:09.005 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:07:09.049 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:07:09.080 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:07:09.144 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:07:09.188 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:07:09.282 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:07:09.330 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:07:09.381 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:07:09.736 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:07:09.805 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:07:09.824 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:07:09.852 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:07:09.860 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:07:09.908 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:07:10.016 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:07:10.052 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:07:10.704 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:07:10.768 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:07:10.842 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:07:11.362 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:07:11.422 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:07:12.291 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:07:12.433 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:07:12.512 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:07:12.520 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:07:12.554 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:07:12.651 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:07:12.694 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:07:12.694 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:07:12.700 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:07:12.803 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:07:12.844 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:07:12.844 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:07:12.875 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:07:12.915 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:07:12.988 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:07:12.988 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:07:12.988 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:07:12.988 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:07:12.988 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:07:12.988 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:12.989 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:07:13.057 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:07:13.094 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:07:13.097 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:07:13.273 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:07:13.354 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:07:13.389 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:07:13.390 INFO:teuthology.orchestra.run.smithi026.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T21:07:13.390 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:14.410 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:07:14.443 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:07:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:07:14.511 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:07:14.511 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:07:14.511 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:07:14.511 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:07:14.511 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:14.569 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:07:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:07:14.636 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:07:14.636 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:07:14.636 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:14.719 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:07:14.795 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:07:14.796 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:07:14.796 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:07:14.796 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:07:14.796 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:07:14.796 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:15.145 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:07:15.219 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:07:15.219 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:07:15.219 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:07:15.219 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:07:15.219 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:07:15.219 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:07:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:07:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:07:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:07:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:07:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:07:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:07:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:07:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:07:19.194 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:07:19.194 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:07:19.194 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:07:19.194 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:07:19.194 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:07:19.194 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:07:19.194 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:07:19.194 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:07:19.198 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:07:19.198 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:07:19.198 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:07:19.198 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:07:19.198 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:07:19.198 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:07:19.198 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:07:19.198 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:07:19.199 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:07:19.210 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:07:19.211 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:07:19.211 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:07:19.211 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:07:19.211 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:07:19.211 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:07:19.211 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:07:19.211 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:07:19.211 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:07:19.211 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:07:19.211 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:07:19.211 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:07:19.211 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:07:19.211 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:07:19.212 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:07:19.212 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:07:19.212 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:07:19.212 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:07:19.212 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:07:19.212 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:07:19.212 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:07:19.212 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:07:19.212 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:07:19.212 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:07:19.212 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:07:19.212 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:07:19.212 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:07:19.213 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:07:19.213 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:07:19.213 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:07:19.213 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:07:19.213 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:07:19.213 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:07:19.213 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:07:19.213 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:07:19.213 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:07:19.213 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:07:19.213 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:07:19.213 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:07:19.213 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:07:19.214 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:07:19.214 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:07:19.214 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:07:19.214 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:07:19.214 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:07:19.214 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:07:19.214 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:07:19.214 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:07:19.214 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:07:19.214 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:07:19.214 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:07:19.214 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:07:19.215 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:07:19.215 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:07:19.215 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:07:19.215 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:07:19.215 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:07:19.215 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:07:19.215 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:07:19.215 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:07:19.215 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:07:19.215 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:07:19.215 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:07:19.215 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:07:19.215 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:07:19.216 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:07:19.216 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:07:19.216 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:07:19.216 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:07:19.216 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:07:19.216 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:07:19.216 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:07:19.216 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:07:19.216 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:07:19.216 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:07:19.216 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:07:19.216 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:07:19.216 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:07:19.216 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:07:19.217 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:07:19.217 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:07:19.217 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:07:19.217 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:07:19.217 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:07:19.217 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:07:19.217 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:07:19.217 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:07:19.217 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:07:19.217 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:07:19.217 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:07:19.217 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:07:19.217 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:07:19.218 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:07:19.218 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:07:19.218 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:07:19.218 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:07:19.218 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:07:19.218 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:07:19.218 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:07:19.218 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:07:19.218 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:07:19.218 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:07:19.218 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:07:19.218 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:07:19.218 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:07:19.219 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:07:19.219 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:07:19.219 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:07:19.219 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:07:19.219 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:07:19.219 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:07:19.219 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:07:19.219 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:07:19.219 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:07:19.219 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:07:19.219 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:07:19.219 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:07:19.219 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:07:19.220 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:07:19.220 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:07:19.220 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:07:19.220 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:07:19.220 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:07:19.220 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:07:19.220 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:07:19.220 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:07:19.220 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:07:19.220 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:07:19.220 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:07:20.278 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:07:20.335 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:07:20.430 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:07:20.498 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:07:20.547 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:07:20.869 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:07:20.930 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:07:20.959 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:07:21.632 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:07:21.633 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:21.633 INFO:teuthology.orchestra.run.smithi026.stdout:Upgraded: 2024-04-26T21:07:21.633 INFO:teuthology.orchestra.run.smithi026.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.633 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.633 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:07:21.633 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:07:21.633 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-04-26T21:07:21.633 INFO:teuthology.orchestra.run.smithi026.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:07:21.633 INFO:teuthology.orchestra.run.smithi026.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:07:21.633 INFO:teuthology.orchestra.run.smithi026.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:07:21.633 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:07:21.633 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.633 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.633 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.634 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.634 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:07:21.634 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.634 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.634 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.634 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:07:21.634 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:07:21.634 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:07:21.634 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:07:21.634 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:07:21.634 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.634 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.635 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:07:21.635 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.635 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.635 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.635 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:07:21.635 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:07:21.635 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.635 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.635 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:07:21.635 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:07:21.635 INFO:teuthology.orchestra.run.smithi026.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:07:21.635 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:07:21.635 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:07:21.635 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:07:21.636 INFO:teuthology.orchestra.run.smithi026.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:07:21.636 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:07:21.636 INFO:teuthology.orchestra.run.smithi026.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:07:21.636 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:07:21.636 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:07:21.636 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:07:21.636 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.636 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.636 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.636 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:07:21.636 INFO:teuthology.orchestra.run.smithi026.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:07:21.636 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:07:21.636 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:07:21.637 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:07:21.637 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.637 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.637 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:07:21.637 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.637 INFO:teuthology.orchestra.run.smithi026.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:07:21.637 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:07:21.637 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:07:21.637 INFO:teuthology.orchestra.run.smithi026.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:07:21.637 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:07:21.637 INFO:teuthology.orchestra.run.smithi026.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:07:21.637 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:07:21.638 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:07:21.638 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:07:21.638 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:07:21.638 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:07:21.638 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:07:21.638 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:07:21.638 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:07:21.638 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:07:21.638 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:07:21.638 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:07:21.638 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:07:21.638 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:07:21.638 INFO:teuthology.orchestra.run.smithi026.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:07:21.639 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:07:21.639 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:07:21.639 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:07:21.639 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.639 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.639 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.639 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:07:21.639 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:07:21.639 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:07:21.639 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:07:21.639 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:07:21.639 INFO:teuthology.orchestra.run.smithi026.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:07:21.639 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:07:21.639 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:07:21.640 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:07:21.640 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:07:21.640 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:07:21.640 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:07:21.640 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:07:21.640 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:07:21.640 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:07:21.640 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:07:21.640 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:07:21.640 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:07:21.640 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:07:21.640 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:07:21.640 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:07:21.641 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:07:21.641 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:07:21.641 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:07:21.641 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:07:21.641 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:07:21.641 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:07:21.641 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:07:21.641 INFO:teuthology.orchestra.run.smithi026.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:07:21.641 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:07:21.641 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:07:21.641 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:07:21.641 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:07:21.641 INFO:teuthology.orchestra.run.smithi026.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:07:21.641 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:07:21.641 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:07:21.642 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:07:21.642 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:07:21.642 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:07:21.642 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:07:21.642 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.642 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.642 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:07:21.642 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:07:21.642 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:07:21.642 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.642 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:07:21.642 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:07:21.643 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:07:21.643 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:07:21.643 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:07:21.643 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:07:21.643 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:07:21.643 INFO:teuthology.orchestra.run.smithi026.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:07:21.643 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:07:21.643 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:07:21.646 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:07:21.646 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:07:21.646 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:07:21.646 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:07:21.646 INFO:teuthology.orchestra.run.smithi026.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:07:21.646 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.646 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.646 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:21.647 INFO:teuthology.orchestra.run.smithi026.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:07:21.647 INFO:teuthology.orchestra.run.smithi026.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:07:21.647 INFO:teuthology.orchestra.run.smithi026.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:07:21.647 INFO:teuthology.orchestra.run.smithi026.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:07:21.647 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:07:21.647 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:07:21.647 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:07:21.647 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:21.648 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-04-26T21:07:21.862 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:07:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:07:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:07:22.322 DEBUG:teuthology.parallel:result is None 2024-04-26T21:07:23.336 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:07:23.446 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:07:23.524 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:07:23.706 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:07:23.752 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:07:23.752 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:07:23.903 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:07:23.951 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:07:23.951 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:07:24.100 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:07:24.229 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:07:24.520 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:07:24.520 INFO:teuthology.orchestra.run.smithi086.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T21:07:24.520 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:30.311 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:07:30.311 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:07:30.311 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:07:30.311 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:07:30.311 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:07:30.312 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:07:30.313 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:07:30.330 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:07:30.330 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:07:30.330 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:07:30.330 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:07:30.330 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:07:30.331 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:07:30.331 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:07:30.331 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:07:30.331 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:07:30.331 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:07:30.331 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:07:30.332 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:07:30.332 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:07:30.332 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:07:30.332 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:07:30.332 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:07:30.332 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:07:30.333 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:07:30.333 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:07:30.333 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:07:30.333 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:07:30.333 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:07:30.333 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:07:30.333 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:07:30.333 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:07:30.334 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:07:30.334 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:07:30.334 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:07:30.334 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:07:30.334 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:07:30.335 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:07:30.335 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:07:30.335 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:07:30.335 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:07:30.335 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:07:30.335 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:07:30.335 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:07:30.336 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:07:30.336 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:07:30.336 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:07:30.336 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:07:30.336 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:07:30.336 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:07:30.337 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:07:30.337 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:07:30.337 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:07:30.337 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:07:30.337 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:07:30.337 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:07:30.338 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:07:30.338 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:07:30.338 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:07:30.338 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:07:30.338 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:07:30.339 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:07:30.339 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:07:30.339 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:07:30.339 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:07:30.339 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:07:30.339 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:07:30.340 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:07:30.340 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:07:30.340 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:07:30.340 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:07:30.340 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:07:30.340 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:07:30.341 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:07:30.341 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:07:30.341 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:07:30.341 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:07:30.341 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:07:30.341 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:07:30.342 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:07:30.342 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:07:30.342 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:07:30.342 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:07:30.342 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:07:30.342 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:07:30.343 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:07:30.343 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:07:30.343 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:07:30.343 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:07:30.343 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:07:30.343 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:07:30.344 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:07:30.344 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:07:30.344 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:07:30.344 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:07:30.344 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:07:30.344 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:07:30.345 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:07:30.345 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:07:30.345 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:07:30.345 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:07:30.345 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:07:30.345 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:07:30.346 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:07:30.346 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:07:30.346 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:07:30.346 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:07:30.346 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:07:30.346 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:07:30.347 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:07:30.347 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:07:30.347 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:07:30.347 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:07:30.347 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:07:30.347 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:07:30.348 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:07:30.348 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:07:30.348 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:07:30.348 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:07:30.348 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:07:30.348 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:07:30.348 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:07:30.349 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:07:30.349 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:07:30.349 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:07:30.349 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:07:30.349 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:07:30.349 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:07:30.349 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:07:30.349 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:07:30.350 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:07:30.350 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:07:32.926 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:07:32.926 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:32.926 INFO:teuthology.orchestra.run.smithi086.stdout:Upgraded: 2024-04-26T21:07:32.926 INFO:teuthology.orchestra.run.smithi086.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.926 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.926 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:07:32.926 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:07:32.926 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-04-26T21:07:32.926 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:07:32.926 INFO:teuthology.orchestra.run.smithi086.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:07:32.926 INFO:teuthology.orchestra.run.smithi086.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:07:32.927 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:07:32.927 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.927 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.927 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.927 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.927 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:07:32.927 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.927 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.927 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.927 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:07:32.927 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:07:32.927 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:07:32.927 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:07:32.927 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:07:32.928 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.928 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.928 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:07:32.928 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.928 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.928 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.928 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:07:32.928 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:07:32.928 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.928 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.928 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:07:32.928 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:07:32.928 INFO:teuthology.orchestra.run.smithi086.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:07:32.929 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:07:32.929 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:07:32.929 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:07:32.929 INFO:teuthology.orchestra.run.smithi086.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:07:32.929 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:07:32.929 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:07:32.929 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:07:32.929 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:07:32.929 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:07:32.929 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.929 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.929 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.929 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:07:32.929 INFO:teuthology.orchestra.run.smithi086.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:07:32.930 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:07:32.930 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:07:32.930 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:07:32.930 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.930 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.930 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:07:32.930 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.930 INFO:teuthology.orchestra.run.smithi086.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:07:32.930 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:07:32.930 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:07:32.930 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:07:32.930 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:07:32.930 INFO:teuthology.orchestra.run.smithi086.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:07:32.931 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:07:32.931 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:07:32.931 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:07:32.931 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:07:32.931 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:07:32.931 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:07:32.931 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:07:32.931 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:07:32.931 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:07:32.931 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:07:32.931 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:07:32.931 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:07:32.931 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:07:32.931 INFO:teuthology.orchestra.run.smithi086.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:07:32.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:07:32.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:07:32.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:07:32.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:07:32.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:07:32.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:07:32.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:07:32.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:07:32.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:07:32.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:07:32.932 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:07:32.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:07:32.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:07:32.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:07:32.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:07:32.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:07:32.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:07:32.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:07:32.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:07:32.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:07:32.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:07:32.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:07:32.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:07:32.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:07:32.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:07:32.933 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:07:32.934 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:07:32.934 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:07:32.934 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:07:32.934 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:07:32.934 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:07:32.934 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:07:32.934 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:07:32.934 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:07:32.934 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:07:32.934 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:07:32.934 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:07:32.934 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:07:32.934 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:07:32.935 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:07:32.935 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:07:32.935 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:07:32.935 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:07:32.935 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.935 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.935 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:07:32.935 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:07:32.935 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:07:32.935 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.935 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:07:32.935 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:07:32.935 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:07:32.935 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:07:32.936 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:07:32.936 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:07:32.936 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:07:32.936 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:07:32.936 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:07:32.936 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:07:32.936 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:07:32.936 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:07:32.936 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:07:32.936 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:07:32.936 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:07:32.936 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.936 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.936 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:07:32.937 INFO:teuthology.orchestra.run.smithi086.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:07:32.937 INFO:teuthology.orchestra.run.smithi086.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:07:32.937 INFO:teuthology.orchestra.run.smithi086.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:07:32.937 INFO:teuthology.orchestra.run.smithi086.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:07:32.937 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:07:32.937 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:07:32.937 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:07:32.937 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:32.937 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-04-26T21:07:33.547 DEBUG:teuthology.parallel:result is None 2024-04-26T21:07:33.548 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:07:33.687 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:07:33.763 INFO:teuthology.orchestra.run.smithi026.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:07:33.764 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:07:33.764 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:07:33.766 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:07:33.907 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:07:33.966 INFO:teuthology.orchestra.run.smithi086.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:07:33.966 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:07:33.966 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:07:33.968 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T21:07:33.968 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:07:33.968 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:07:34.072 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:07:34.073 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:07:34.150 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T21:07:34.151 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:07:34.151 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:07:34.244 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:07:34.375 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:07:34.376 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:07:34.442 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:07:34.549 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T21:07:34.550 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:07:34.550 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:07:34.644 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:07:34.775 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:07:34.775 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:07:34.853 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:07:34.961 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T21:07:34.961 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:07:34.961 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:07:35.052 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:07:35.140 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:07:35.162 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:07:35.230 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:07:35.342 INFO:teuthology.run_tasks:Running task ceph... 2024-04-26T21:07:35.432 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-26T21:07:35.432 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 777 /var/log/ceph 2024-04-26T21:07:35.440 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /var/log/ceph 2024-04-26T21:07:35.525 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-26T21:07:35.525 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T21:07:35.530 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T21:07:35.611 INFO:tasks.ceph:Creating extra log directories... 2024-04-26T21:07:35.612 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T21:07:35.617 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T21:07:35.711 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-26T21:07:35.711 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-26T21:07:35.711 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-26T21:07:35.711 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:07:35.711 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T21:07:35.769 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T21:07:36.004 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T21:07:36.093 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:07:36.093 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T21:07:36.139 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T21:07:36.322 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T21:07:36.391 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-26T21:07:36.391 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-26T21:07:36.391 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675194', 'branch': 'wip-rishabh-testing-20240426.111959', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/backtrace}', 'email': None, 'first_in_suite': False, 'job_id': '7675194', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240426.111959'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240426.111959', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 816, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'sleep_before_teardown': 0, 'subset': '166/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240426.111959', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi026.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN+3caLPLKkWOO8ZXqexY8z9NGh/HH5hyG6O6NGFgQIsSNb1vK3UAgFN+F4JVeuAIvzIIqBo0PW8Hs6qaSmImc8=', 'smithi086.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK0wPYqHDPyXwXdZ3vmTmk7/fzwGtg5oeDPlTsCRpzLiKXuAwck96k8Sl4lQrPI6TYMDb1xH6PoiFhNCIAUjjds='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_backtrace']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-26_19:30:57', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-26T21:07:36.392 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T21:07:36.400 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T21:07:36.450 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T21:07:36.457 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T21:07:36.541 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:07:36.542 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:07:36.600 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:07:36.600 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_1 2024-04-26T21:07:36.654 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:07:36.654 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:07:36.654 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-26T21:07:36.654 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:07:36.654 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:07:36.654 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-04-26 21:07:13.908503644 +0000 2024-04-26T21:07:36.654 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-04-26 21:04:43.374295623 +0000 2024-04-26T21:07:36.654 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-04-26 21:04:43.374295623 +0000 2024-04-26T21:07:36.654 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-04-26 21:04:43.374295623 +0000 2024-04-26T21:07:36.655 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:07:36.739 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-04-26T21:07:36.739 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-04-26T21:07:36.739 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000307512 s, 1.7 MB/s 2024-04-26T21:07:36.743 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:07:36.805 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_2 2024-04-26T21:07:36.859 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:07:36.859 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:07:36.860 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-26T21:07:36.860 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:07:36.860 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:07:36.860 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-04-26 21:07:13.910503614 +0000 2024-04-26T21:07:36.860 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-04-26 21:04:43.977286398 +0000 2024-04-26T21:07:36.860 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-04-26 21:04:43.977286398 +0000 2024-04-26T21:07:36.860 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-04-26 21:04:43.977286398 +0000 2024-04-26T21:07:36.860 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:07:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-04-26T21:07:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-04-26T21:07:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000260217 s, 2.0 MB/s 2024-04-26T21:07:36.948 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:07:37.010 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_3 2024-04-26T21:07:37.060 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:07:37.061 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:07:37.061 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-26T21:07:37.061 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:07:37.061 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:07:37.061 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-04-26 21:07:13.910503614 +0000 2024-04-26T21:07:37.061 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-04-26 21:04:44.595276944 +0000 2024-04-26T21:07:37.061 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-04-26 21:04:44.595276944 +0000 2024-04-26T21:07:37.061 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-04-26 21:04:44.595276944 +0000 2024-04-26T21:07:37.061 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:07:37.136 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-04-26T21:07:37.136 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-04-26T21:07:37.137 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000277086 s, 1.8 MB/s 2024-04-26T21:07:37.140 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:07:37.198 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_4 2024-04-26T21:07:37.252 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:07:37.252 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:07:37.252 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-04-26T21:07:37.252 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:07:37.252 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:07:37.252 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-04-26 21:07:13.911503599 +0000 2024-04-26T21:07:37.252 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-04-26 21:04:45.160268301 +0000 2024-04-26T21:07:37.252 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-04-26 21:04:45.160268301 +0000 2024-04-26T21:07:37.252 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-04-26 21:04:45.160268301 +0000 2024-04-26T21:07:37.252 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:07:37.334 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-04-26T21:07:37.334 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-04-26T21:07:37.334 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000254933 s, 2.0 MB/s 2024-04-26T21:07:37.338 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:07:37.401 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:07:37.401 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:07:37.401 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:07:37.445 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:07:37.446 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2024-04-26T21:07:37.530 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:07:37.530 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:07:37.530 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-04-26T21:07:37.530 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:07:37.530 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:07:37.530 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-26 21:07:25.023478806 +0000 2024-04-26T21:07:37.530 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-26 21:04:38.764362238 +0000 2024-04-26T21:07:37.530 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-26 21:04:38.764362238 +0000 2024-04-26T21:07:37.530 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-26 21:04:38.764362238 +0000 2024-04-26T21:07:37.531 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:07:37.628 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-26T21:07:37.628 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-26T21:07:37.628 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000379903 s, 1.3 MB/s 2024-04-26T21:07:37.631 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:07:37.679 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2024-04-26T21:07:37.721 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:07:37.721 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:07:37.721 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-04-26T21:07:37.722 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:07:37.722 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:07:37.722 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-26 21:07:25.024478785 +0000 2024-04-26T21:07:37.722 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-26 21:04:39.377348583 +0000 2024-04-26T21:07:37.722 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-26 21:04:39.377348583 +0000 2024-04-26T21:07:37.722 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-26 21:04:39.377348583 +0000 2024-04-26T21:07:37.722 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:07:37.832 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-26T21:07:37.832 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-26T21:07:37.832 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.00033925 s, 1.5 MB/s 2024-04-26T21:07:37.836 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:07:37.881 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2024-04-26T21:07:37.958 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:07:37.958 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:07:37.959 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-26T21:07:37.959 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:07:37.959 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:07:37.959 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-26 21:07:25.025478763 +0000 2024-04-26T21:07:37.959 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-26 21:04:39.969335393 +0000 2024-04-26T21:07:37.959 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-26 21:04:39.969335393 +0000 2024-04-26T21:07:37.959 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-26 21:04:39.969335393 +0000 2024-04-26T21:07:37.959 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:07:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-26T21:07:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-26T21:07:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000343974 s, 1.5 MB/s 2024-04-26T21:07:38.072 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:07:38.117 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2024-04-26T21:07:38.207 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:07:38.208 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:07:38.208 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-04-26T21:07:38.208 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:07:38.208 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:07:38.208 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-26 21:07:25.026478741 +0000 2024-04-26T21:07:38.208 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-26 21:04:40.545322559 +0000 2024-04-26T21:07:38.208 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-26 21:04:40.545322559 +0000 2024-04-26T21:07:38.208 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-26 21:04:40.545322559 +0000 2024-04-26T21:07:38.209 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:07:38.269 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-26T21:07:38.269 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-26T21:07:38.269 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.00025391 s, 2.0 MB/s 2024-04-26T21:07:38.272 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:07:38.312 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:07:38.312 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi026.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi086.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T21:07:38.312 INFO:tasks.ceph:Generating config... 2024-04-26T21:07:38.313 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-26T21:07:38.314 INFO:tasks.ceph:[client] debug client = 20 2024-04-26T21:07:38.314 INFO:tasks.ceph:[client] debug ms = 1 2024-04-26T21:07:38.314 INFO:tasks.ceph:[client] pid file = 2024-04-26T21:07:38.314 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-26T21:07:38.314 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-26T21:07:38.314 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-26T21:07:38.314 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-26T21:07:38.314 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-26T21:07:38.314 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-26T21:07:38.314 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-26T21:07:38.314 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-26T21:07:38.315 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-26T21:07:38.315 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-26T21:07:38.315 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-26T21:07:38.315 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-26T21:07:38.315 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-26T21:07:38.315 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-26T21:07:38.315 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-26T21:07:38.315 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-26T21:07:38.315 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-26T21:07:38.315 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-26T21:07:38.316 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-26T21:07:38.316 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-26T21:07:38.316 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-26T21:07:38.316 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-26T21:07:38.316 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-26T21:07:38.316 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-26T21:07:38.316 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-26T21:07:38.316 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-26T21:07:38.316 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-26T21:07:38.317 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-26T21:07:38.317 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-26T21:07:38.317 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-26T21:07:38.317 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-26T21:07:38.317 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-26T21:07:38.317 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-26T21:07:38.317 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-26T21:07:38.317 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-26T21:07:38.317 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-26T21:07:38.317 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-26T21:07:38.318 INFO:tasks.ceph:Setting up mon.a... 2024-04-26T21:07:38.318 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-26T21:07:38.464 INFO:teuthology.orchestra.run.smithi026.stdout:creating /etc/ceph/ceph.keyring 2024-04-26T21:07:38.473 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-26T21:07:38.607 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T21:07:38.701 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.26'), ('mon.b', '172.21.15.86'), ('mon.c', '[v2:172.21.15.86:3301,v1:172.21.15.86:6790]')] 2024-04-26T21:07:38.701 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.26,172.21.15.86,[v2:172.21.15.86:3301,v1:172.21.15.86:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-26T21:07:38.702 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:07:38.703 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T21:07:38.762 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.26 --add b 172.21.15.86 --addv c '[v2:172.21.15.86:3301,v1:172.21.15.86:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:07:38.923 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:07:38.923 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool: generated fsid eaf60df9-83a0-4283-ab2c-83d2c02a2067 2024-04-26T21:07:38.923 INFO:teuthology.orchestra.run.smithi026.stdout:setting min_mon_release = quincy 2024-04-26T21:07:38.923 INFO:teuthology.orchestra.run.smithi026.stdout:epoch 0 2024-04-26T21:07:38.923 INFO:teuthology.orchestra.run.smithi026.stdout:fsid eaf60df9-83a0-4283-ab2c-83d2c02a2067 2024-04-26T21:07:38.923 INFO:teuthology.orchestra.run.smithi026.stdout:last_changed 2024-04-26T21:07:38.922852+0000 2024-04-26T21:07:38.923 INFO:teuthology.orchestra.run.smithi026.stdout:created 2024-04-26T21:07:38.922852+0000 2024-04-26T21:07:38.923 INFO:teuthology.orchestra.run.smithi026.stdout:min_mon_release 17 (quincy) 2024-04-26T21:07:38.923 INFO:teuthology.orchestra.run.smithi026.stdout:election_strategy: 1 2024-04-26T21:07:38.923 INFO:teuthology.orchestra.run.smithi026.stdout:0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-04-26T21:07:38.923 INFO:teuthology.orchestra.run.smithi026.stdout:1: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.b 2024-04-26T21:07:38.924 INFO:teuthology.orchestra.run.smithi026.stdout:2: [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] mon.c 2024-04-26T21:07:38.924 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-26T21:07:38.926 DEBUG:teuthology.orchestra.run.smithi026:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T21:07:38.978 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID eaf60df9-83a0-4283-ab2c-83d2c02a2067... 2024-04-26T21:07:38.979 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T21:07:38.986 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T21:07:39.097 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-04-26T21:07:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: chdir = "" 2024-04-26T21:07:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T21:07:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: auth supported = cephx 2024-04-26T21:07:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: filestore xattr use omap = true 2024-04-26T21:07:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: mon clock drift allowed = 1.000 2024-04-26T21:07:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: osd crush chooseleaf type = 0 2024-04-26T21:07:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: auth debug = true 2024-04-26T21:07:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on old message = true 2024-04-26T21:07:39.098 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on bug = true 2024-04-26T21:07:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T21:07:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: mon pg warn max object skew = 0 2024-04-26T21:07:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T21:07:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T21:07:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default size = 2 2024-04-26T21:07:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow primary affinity = true 2024-04-26T21:07:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow pg remap = true 2024-04-26T21:07:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on legacy crush tunables = false 2024-04-26T21:07:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on crush straw calc version zero = false 2024-04-26T21:07:39.099 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on no sortbitwise = false 2024-04-26T21:07:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on osd down out interval zero = false 2024-04-26T21:07:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on too few osds = false 2024-04-26T21:07:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T21:07:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T21:07:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: mon_allow_pool_size_one = true 2024-04-26T21:07:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T21:07:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: osd default data pool replay window = 5 2024-04-26T21:07:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: mon allow pool delete = true 2024-04-26T21:07:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: mon cluster log file level = debug 2024-04-26T21:07:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: debug asserts on shutdown = true 2024-04-26T21:07:39.100 INFO:teuthology.orchestra.run.smithi086.stdout: mon health detail to clog = false 2024-04-26T21:07:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: mon host = "172.21.15.26,172.21.15.86,[v2:172.21.15.86:3301,v1:172.21.15.86:6790]" 2024-04-26T21:07:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on skipped message = False 2024-04-26T21:07:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = eaf60df9-83a0-4283-ab2c-83d2c02a2067 2024-04-26T21:07:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.101 INFO:teuthology.orchestra.run.smithi086.stdout:[osd] 2024-04-26T21:07:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: osd journal size = 100 2024-04-26T21:07:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub load threshold = 5.0 2024-04-26T21:07:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub max interval = 600 2024-04-26T21:07:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: osd mclock profile = high_recovery_ops 2024-04-26T21:07:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: osd recover clone overlap = true 2024-04-26T21:07:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: osd recovery max chunk = 1048576 2024-04-26T21:07:39.101 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug shutdown = true 2024-04-26T21:07:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug op order = true 2024-04-26T21:07:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug verify stray on activate = true 2024-04-26T21:07:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug trim objects = true 2024-04-26T21:07:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: osd open classes on start = true 2024-04-26T21:07:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug pg log writeout = true 2024-04-26T21:07:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: osd deep scrub update digest min age = 30 2024-04-26T21:07:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: osd map max advance = 10 2024-04-26T21:07:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: journal zero on create = true 2024-04-26T21:07:39.102 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: filestore ondisk finisher threads = 3 2024-04-26T21:07:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: filestore apply finisher threads = 3 2024-04-26T21:07:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: bdev debug aio = true 2024-04-26T21:07:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug misdirected ops = true 2024-04-26T21:07:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: bdev async discard = True 2024-04-26T21:07:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: bdev enable discard = True 2024-04-26T21:07:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore allocator = bitmap 2024-04-26T21:07:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore block size = 96636764160 2024-04-26T21:07:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore fsck on mount = True 2024-04-26T21:07:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluefs = 1/20 2024-04-26T21:07:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluestore = 1/20 2024-04-26T21:07:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-26T21:07:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: debug osd = 20 2024-04-26T21:07:39.103 INFO:teuthology.orchestra.run.smithi086.stdout: debug rocksdb = 4/10 2024-04-26T21:07:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T21:07:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd full ratio = 0.9 2024-04-26T21:07:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd nearfull ratio = 0.8 2024-04-26T21:07:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: osd failsafe full ratio = 0.95 2024-04-26T21:07:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: osd objectstore = bluestore 2024-04-26T21:07:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-04-26T21:07:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.104 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr] 2024-04-26T21:07:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-26T21:07:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: debug mgr = 20 2024-04-26T21:07:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-04-26T21:07:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-04-26T21:07:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-04-26T21:07:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-04-26T21:07:39.104 INFO:teuthology.orchestra.run.smithi086.stdout: mgr/telemetry/nag = false 2024-04-26T21:07:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-04-26T21:07:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.105 INFO:teuthology.orchestra.run.smithi086.stdout:[mon] 2024-04-26T21:07:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-26T21:07:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-04-26T21:07:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: debug paxos = 20 2024-04-26T21:07:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-04-26T21:07:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: mon data avail warn = 5 2024-04-26T21:07:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: mon mgr mkfs grace = 240 2024-04-26T21:07:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-04-26T21:07:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd reporter subtree level = osd 2024-04-26T21:07:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd prime pg temp = true 2024-04-26T21:07:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-04-26T21:07:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.105 INFO:teuthology.orchestra.run.smithi086.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T21:07:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T21:07:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T21:07:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T21:07:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T21:07:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T21:07:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: # 1m isn't quite enough 2024-04-26T21:07:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: mon_down_mkfs_grace = 2m 2024-04-26T21:07:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_filestore_osds = false 2024-04-26T21:07:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: mon down mkfs grace = 300 2024-04-26T21:07:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: mon op complaint time = 120 2024-04-26T21:07:39.106 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-26T21:07:39.106 INFO:teuthology.orchestra.run.smithi086.stdout:[client] 2024-04-26T21:07:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: rgw cache enabled = true 2024-04-26T21:07:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable ops log = true 2024-04-26T21:07:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable usage log = true 2024-04-26T21:07:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T21:07:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T21:07:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: client mount timeout = 600 2024-04-26T21:07:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-04-26T21:07:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-26T21:07:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = "" 2024-04-26T21:07:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-04-26T21:07:39.107 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-04-26T21:07:39.107 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.a] 2024-04-26T21:07:39.107 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.b] 2024-04-26T21:07:39.107 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.c] 2024-04-26T21:07:39.107 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2024-04-26T21:07:39.108 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2024-04-26T21:07:39.108 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2024-04-26T21:07:39.108 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2024-04-26T21:07:39.108 INFO:teuthology.orchestra.run.smithi086.stdout:[mds] 2024-04-26T21:07:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds = 20 2024-04-26T21:07:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds balancer = 20 2024-04-26T21:07:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-26T21:07:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug frag = True 2024-04-26T21:07:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug scatterstat = True 2024-04-26T21:07:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: mds op complaint time = 180 2024-04-26T21:07:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: mds verify scatter = True 2024-04-26T21:07:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-04-26T21:07:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-04-26T21:07:39.108 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-04-26T21:07:39.128 INFO:teuthology.orchestra.run.smithi026.stdout:[global] 2024-04-26T21:07:39.128 INFO:teuthology.orchestra.run.smithi026.stdout: chdir = "" 2024-04-26T21:07:39.128 INFO:teuthology.orchestra.run.smithi026.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T21:07:39.128 INFO:teuthology.orchestra.run.smithi026.stdout: auth supported = cephx 2024-04-26T21:07:39.128 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.128 INFO:teuthology.orchestra.run.smithi026.stdout: filestore xattr use omap = true 2024-04-26T21:07:39.128 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.128 INFO:teuthology.orchestra.run.smithi026.stdout: mon clock drift allowed = 1.000 2024-04-26T21:07:39.129 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.129 INFO:teuthology.orchestra.run.smithi026.stdout: osd crush chooseleaf type = 0 2024-04-26T21:07:39.129 INFO:teuthology.orchestra.run.smithi026.stdout: auth debug = true 2024-04-26T21:07:39.129 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.129 INFO:teuthology.orchestra.run.smithi026.stdout: ms die on old message = true 2024-04-26T21:07:39.129 INFO:teuthology.orchestra.run.smithi026.stdout: ms die on bug = true 2024-04-26T21:07:39.129 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.129 INFO:teuthology.orchestra.run.smithi026.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T21:07:39.129 INFO:teuthology.orchestra.run.smithi026.stdout: mon pg warn max object skew = 0 2024-04-26T21:07:39.129 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.129 INFO:teuthology.orchestra.run.smithi026.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T21:07:39.129 INFO:teuthology.orchestra.run.smithi026.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T21:07:39.129 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.129 INFO:teuthology.orchestra.run.smithi026.stdout: osd pool default size = 2 2024-04-26T21:07:39.129 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.130 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd allow primary affinity = true 2024-04-26T21:07:39.130 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd allow pg remap = true 2024-04-26T21:07:39.130 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on legacy crush tunables = false 2024-04-26T21:07:39.130 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on crush straw calc version zero = false 2024-04-26T21:07:39.130 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on no sortbitwise = false 2024-04-26T21:07:39.130 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on osd down out interval zero = false 2024-04-26T21:07:39.130 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on too few osds = false 2024-04-26T21:07:39.130 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T21:07:39.130 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T21:07:39.130 INFO:teuthology.orchestra.run.smithi026.stdout: mon_allow_pool_size_one = true 2024-04-26T21:07:39.130 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.130 INFO:teuthology.orchestra.run.smithi026.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T21:07:39.130 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.131 INFO:teuthology.orchestra.run.smithi026.stdout: osd default data pool replay window = 5 2024-04-26T21:07:39.131 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.131 INFO:teuthology.orchestra.run.smithi026.stdout: mon allow pool delete = true 2024-04-26T21:07:39.131 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.131 INFO:teuthology.orchestra.run.smithi026.stdout: mon cluster log file level = debug 2024-04-26T21:07:39.131 INFO:teuthology.orchestra.run.smithi026.stdout: debug asserts on shutdown = true 2024-04-26T21:07:39.131 INFO:teuthology.orchestra.run.smithi026.stdout: mon health detail to clog = false 2024-04-26T21:07:39.131 INFO:teuthology.orchestra.run.smithi026.stdout: mon host = "172.21.15.26,172.21.15.86,[v2:172.21.15.86:3301,v1:172.21.15.86:6790]" 2024-04-26T21:07:39.131 INFO:teuthology.orchestra.run.smithi026.stdout: ms die on skipped message = False 2024-04-26T21:07:39.131 INFO:teuthology.orchestra.run.smithi026.stdout: fsid = eaf60df9-83a0-4283-ab2c-83d2c02a2067 2024-04-26T21:07:39.131 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.131 INFO:teuthology.orchestra.run.smithi026.stdout:[osd] 2024-04-26T21:07:39.131 INFO:teuthology.orchestra.run.smithi026.stdout: osd journal size = 100 2024-04-26T21:07:39.131 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.131 INFO:teuthology.orchestra.run.smithi026.stdout: osd scrub load threshold = 5.0 2024-04-26T21:07:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: osd scrub max interval = 600 2024-04-26T21:07:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: osd mclock profile = high_recovery_ops 2024-04-26T21:07:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: osd recover clone overlap = true 2024-04-26T21:07:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: osd recovery max chunk = 1048576 2024-04-26T21:07:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug shutdown = true 2024-04-26T21:07:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug op order = true 2024-04-26T21:07:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug verify stray on activate = true 2024-04-26T21:07:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug trim objects = true 2024-04-26T21:07:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: osd open classes on start = true 2024-04-26T21:07:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug pg log writeout = true 2024-04-26T21:07:39.132 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.133 INFO:teuthology.orchestra.run.smithi026.stdout: osd deep scrub update digest min age = 30 2024-04-26T21:07:39.133 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.133 INFO:teuthology.orchestra.run.smithi026.stdout: osd map max advance = 10 2024-04-26T21:07:39.133 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.133 INFO:teuthology.orchestra.run.smithi026.stdout: journal zero on create = true 2024-04-26T21:07:39.133 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.133 INFO:teuthology.orchestra.run.smithi026.stdout: filestore ondisk finisher threads = 3 2024-04-26T21:07:39.133 INFO:teuthology.orchestra.run.smithi026.stdout: filestore apply finisher threads = 3 2024-04-26T21:07:39.133 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.133 INFO:teuthology.orchestra.run.smithi026.stdout: bdev debug aio = true 2024-04-26T21:07:39.133 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug misdirected ops = true 2024-04-26T21:07:39.133 INFO:teuthology.orchestra.run.smithi026.stdout: bdev async discard = True 2024-04-26T21:07:39.133 INFO:teuthology.orchestra.run.smithi026.stdout: bdev enable discard = True 2024-04-26T21:07:39.133 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore allocator = bitmap 2024-04-26T21:07:39.133 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore block size = 96636764160 2024-04-26T21:07:39.134 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore fsck on mount = True 2024-04-26T21:07:39.134 INFO:teuthology.orchestra.run.smithi026.stdout: debug bluefs = 1/20 2024-04-26T21:07:39.134 INFO:teuthology.orchestra.run.smithi026.stdout: debug bluestore = 1/20 2024-04-26T21:07:39.134 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-04-26T21:07:39.134 INFO:teuthology.orchestra.run.smithi026.stdout: debug osd = 20 2024-04-26T21:07:39.134 INFO:teuthology.orchestra.run.smithi026.stdout: debug rocksdb = 4/10 2024-04-26T21:07:39.134 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T21:07:39.134 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd full ratio = 0.9 2024-04-26T21:07:39.134 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd nearfull ratio = 0.8 2024-04-26T21:07:39.134 INFO:teuthology.orchestra.run.smithi026.stdout: osd failsafe full ratio = 0.95 2024-04-26T21:07:39.134 INFO:teuthology.orchestra.run.smithi026.stdout: osd objectstore = bluestore 2024-04-26T21:07:39.134 INFO:teuthology.orchestra.run.smithi026.stdout: osd op complaint time = 180 2024-04-26T21:07:39.134 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.134 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr] 2024-04-26T21:07:39.135 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-04-26T21:07:39.135 INFO:teuthology.orchestra.run.smithi026.stdout: debug mgr = 20 2024-04-26T21:07:39.135 INFO:teuthology.orchestra.run.smithi026.stdout: debug mon = 20 2024-04-26T21:07:39.135 INFO:teuthology.orchestra.run.smithi026.stdout: debug auth = 20 2024-04-26T21:07:39.135 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min pgs per osd = 4 2024-04-26T21:07:39.135 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min bytes per osd = 10 2024-04-26T21:07:39.135 INFO:teuthology.orchestra.run.smithi026.stdout: mgr/telemetry/nag = false 2024-04-26T21:07:39.135 INFO:teuthology.orchestra.run.smithi026.stdout: debug client = 20 2024-04-26T21:07:39.135 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.135 INFO:teuthology.orchestra.run.smithi026.stdout:[mon] 2024-04-26T21:07:39.135 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-04-26T21:07:39.135 INFO:teuthology.orchestra.run.smithi026.stdout: debug mon = 20 2024-04-26T21:07:39.135 INFO:teuthology.orchestra.run.smithi026.stdout: debug paxos = 20 2024-04-26T21:07:39.135 INFO:teuthology.orchestra.run.smithi026.stdout: debug auth = 20 2024-04-26T21:07:39.135 INFO:teuthology.orchestra.run.smithi026.stdout: mon data avail warn = 5 2024-04-26T21:07:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: mon mgr mkfs grace = 240 2024-04-26T21:07:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min pgs per osd = 4 2024-04-26T21:07:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd reporter subtree level = osd 2024-04-26T21:07:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd prime pg temp = true 2024-04-26T21:07:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min bytes per osd = 10 2024-04-26T21:07:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T21:07:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T21:07:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T21:07:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T21:07:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T21:07:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T21:07:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: # 1m isn't quite enough 2024-04-26T21:07:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: mon_down_mkfs_grace = 2m 2024-04-26T21:07:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_filestore_osds = false 2024-04-26T21:07:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: mon down mkfs grace = 300 2024-04-26T21:07:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: mon op complaint time = 120 2024-04-26T21:07:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:07:39.137 INFO:teuthology.orchestra.run.smithi026.stdout:[client] 2024-04-26T21:07:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: rgw cache enabled = true 2024-04-26T21:07:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: rgw enable ops log = true 2024-04-26T21:07:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: rgw enable usage log = true 2024-04-26T21:07:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T21:07:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T21:07:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: client mount timeout = 600 2024-04-26T21:07:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: debug client = 20 2024-04-26T21:07:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-04-26T21:07:39.138 INFO:teuthology.orchestra.run.smithi026.stdout: pid file = "" 2024-04-26T21:07:39.138 INFO:teuthology.orchestra.run.smithi026.stdout: rados mon op timeout = 900 2024-04-26T21:07:39.138 INFO:teuthology.orchestra.run.smithi026.stdout: rados osd op timeout = 900 2024-04-26T21:07:39.138 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.a] 2024-04-26T21:07:39.138 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.b] 2024-04-26T21:07:39.138 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.c] 2024-04-26T21:07:39.138 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.a] 2024-04-26T21:07:39.138 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.c] 2024-04-26T21:07:39.138 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.b] 2024-04-26T21:07:39.138 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.d] 2024-04-26T21:07:39.138 INFO:teuthology.orchestra.run.smithi026.stdout:[mds] 2024-04-26T21:07:39.138 INFO:teuthology.orchestra.run.smithi026.stdout: debug mds = 20 2024-04-26T21:07:39.138 INFO:teuthology.orchestra.run.smithi026.stdout: debug mds balancer = 20 2024-04-26T21:07:39.138 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-04-26T21:07:39.138 INFO:teuthology.orchestra.run.smithi026.stdout: mds debug frag = True 2024-04-26T21:07:39.139 INFO:teuthology.orchestra.run.smithi026.stdout: mds debug scatterstat = True 2024-04-26T21:07:39.139 INFO:teuthology.orchestra.run.smithi026.stdout: mds op complaint time = 180 2024-04-26T21:07:39.139 INFO:teuthology.orchestra.run.smithi026.stdout: mds verify scatter = True 2024-04-26T21:07:39.139 INFO:teuthology.orchestra.run.smithi026.stdout: osd op complaint time = 180 2024-04-26T21:07:39.139 INFO:teuthology.orchestra.run.smithi026.stdout: rados mon op timeout = 900 2024-04-26T21:07:39.139 INFO:teuthology.orchestra.run.smithi026.stdout: rados osd op timeout = 900 2024-04-26T21:07:39.165 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-26T21:07:39.165 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-26T21:07:39.308 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-26T21:07:39.308 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:07:39.308 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-26T21:07:39.364 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:07:39.364 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-26T21:07:39.419 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi026.front.sepia.ceph.com 2024-04-26T21:07:39.419 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:07:39.419 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T21:07:39.419 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T21:07:39.553 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:07:39.554 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:07:39.654 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi086.front.sepia.ceph.com 2024-04-26T21:07:39.654 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:07:39.654 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T21:07:39.654 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T21:07:39.743 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:07:39.743 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:07:39.825 INFO:tasks.ceph:Setting up mon nodes... 2024-04-26T21:07:39.825 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-26T21:07:39.825 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T21:07:39.989 INFO:teuthology.orchestra.run.smithi026.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T21:07:40.000 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T21:07:40.134 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T21:07:40.142 INFO:tasks.ceph:Setting up mds nodes... 2024-04-26T21:07:40.142 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T21:07:40.304 INFO:teuthology.orchestra.run.smithi026.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T21:07:40.310 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-26T21:07:40.390 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T21:07:40.535 INFO:teuthology.orchestra.run.smithi026.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T21:07:40.542 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-26T21:07:40.620 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T21:07:40.770 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T21:07:40.779 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-26T21:07:40.857 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T21:07:40.992 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T21:07:41.001 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-26T21:07:41.069 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-26T21:07:41.069 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T21:07:41.184 INFO:teuthology.orchestra.run.smithi026.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-26T21:07:41.218 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-26T21:07:41.328 INFO:teuthology.orchestra.run.smithi026.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-26T21:07:41.381 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-26T21:07:41.498 INFO:teuthology.orchestra.run.smithi086.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-26T21:07:41.544 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-26T21:07:41.646 INFO:teuthology.orchestra.run.smithi086.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-26T21:07:41.684 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-26T21:07:41.684 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi026.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi086.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T21:07:41.684 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-26T21:07:41.770 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:07:41.770 INFO:tasks.ceph:role: osd.0 2024-04-26T21:07:41.770 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi026.front.sepia.ceph.com 2024-04-26T21:07:41.770 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T21:07:41.838 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:07:41.838 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:07:41.838 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:07:41.838 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:07:41.838 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:07:41.838 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:07:41.838 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:07:41.838 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:07:41.838 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:07:41.838 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:07:41.872 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-04-26T21:07:41.877 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-04-26T21:07:41.877 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-26T21:07:41.966 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-26T21:07:42.046 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-26T21:07:42.112 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:07:42.112 INFO:tasks.ceph:role: osd.1 2024-04-26T21:07:42.112 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi026.front.sepia.ceph.com 2024-04-26T21:07:42.112 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T21:07:42.185 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:07:42.186 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:07:42.186 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:07:42.186 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:07:42.186 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:07:42.186 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:07:42.186 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:07:42.186 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:07:42.186 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:07:42.186 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:07:42.220 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-04-26T21:07:42.226 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-04-26T21:07:42.226 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-26T21:07:42.334 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-26T21:07:42.419 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-26T21:07:42.490 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:07:42.490 INFO:tasks.ceph:role: osd.2 2024-04-26T21:07:42.490 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi026.front.sepia.ceph.com 2024-04-26T21:07:42.490 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T21:07:42.564 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:07:42.564 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:07:42.564 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:07:42.564 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:07:42.564 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:07:42.565 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:07:42.565 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:07:42.565 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:07:42.565 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:07:42.565 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:07:42.598 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-04-26T21:07:42.604 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-04-26T21:07:42.604 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-26T21:07:42.704 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-26T21:07:42.780 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-26T21:07:42.862 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T21:07:42.862 INFO:tasks.ceph:role: osd.3 2024-04-26T21:07:42.862 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi026.front.sepia.ceph.com 2024-04-26T21:07:42.862 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T21:07:42.950 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:07:42.951 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:07:42.951 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:07:42.951 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:07:42.951 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:07:42.951 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:07:42.951 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:07:42.951 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:07:42.951 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:07:42.951 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:07:42.984 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-04-26T21:07:42.991 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-04-26T21:07:42.991 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-26T21:07:43.107 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-26T21:07:43.181 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:07:43.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:43.328+0000 7fa9440fd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-26T21:07:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:43.328+0000 7fa9440fd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-26T21:07:43.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:43.328+0000 7fa9440fd640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T21:07:43.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:43.328+0000 7fa9440fd640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T21:07:43.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:43.328+0000 7fa9440fd640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T21:07:43.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:43.332+0000 7fa9440fd640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-26T21:07:46.549 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T21:07:46.640 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:07:46.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:46.779+0000 7f44a3a14640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-26T21:07:46.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:46.780+0000 7f44a3a14640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-26T21:07:46.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:46.780+0000 7f44a3a14640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T21:07:46.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:46.780+0000 7f44a3a14640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T21:07:46.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:46.780+0000 7f44a3a14640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T21:07:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:46.782+0000 7f44a3a14640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-26T21:07:49.771 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T21:07:49.859 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:07:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:49.993+0000 7f6d40a30640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-26T21:07:49.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:49.994+0000 7f6d40a30640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-26T21:07:49.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:49.994+0000 7f6d40a30640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T21:07:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:49.994+0000 7f6d40a30640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T21:07:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:49.994+0000 7f6d40a30640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T21:07:49.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:49.996+0000 7f6d40a30640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-26T21:07:53.250 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T21:07:53.334 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:07:53.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:53.464+0000 7f2b70c18640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-26T21:07:53.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:53.465+0000 7f2b70c18640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-26T21:07:53.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:53.465+0000 7f2b70c18640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T21:07:53.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:53.465+0000 7f2b70c18640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T21:07:53.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:53.465+0000 7f2b70c18640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T21:07:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:07:53.467+0000 7f2b70c18640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-26T21:07:56.685 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T21:07:56.776 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-26T21:07:56.866 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:07:56.866 INFO:tasks.ceph:role: osd.4 2024-04-26T21:07:56.866 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-26T21:07:56.866 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T21:07:56.951 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:07:56.951 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:07:56.951 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:07:56.951 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:07:56.951 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:07:56.951 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:07:56.951 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:07:56.951 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:07:56.951 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:07:56.952 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:07:57.364 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-04-26T21:07:57.371 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-04-26T21:07:57.371 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-26T21:07:57.506 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-26T21:07:57.580 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-26T21:07:57.646 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:07:57.646 INFO:tasks.ceph:role: osd.5 2024-04-26T21:07:57.646 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-26T21:07:57.647 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T21:07:57.723 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:07:57.723 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:07:57.723 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:07:57.723 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:07:57.723 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:07:57.723 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:07:57.723 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:07:57.723 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:07:57.723 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:07:57.723 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:07:58.078 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-04-26T21:07:58.083 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-04-26T21:07:58.083 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-26T21:07:58.185 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-26T21:07:58.252 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-26T21:07:58.311 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:07:58.312 INFO:tasks.ceph:role: osd.6 2024-04-26T21:07:58.312 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-26T21:07:58.312 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T21:07:58.383 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:07:58.383 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:07:58.383 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:07:58.383 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:07:58.383 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:07:58.383 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:07:58.383 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:07:58.383 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:07:58.383 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:07:58.383 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:07:58.768 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-04-26T21:07:58.776 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-04-26T21:07:58.776 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-26T21:07:58.888 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-26T21:07:58.961 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-26T21:07:59.032 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T21:07:59.032 INFO:tasks.ceph:role: osd.7 2024-04-26T21:07:59.033 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-26T21:07:59.033 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T21:07:59.116 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T21:07:59.117 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T21:07:59.117 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T21:07:59.117 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T21:07:59.117 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T21:07:59.117 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-04-26T21:07:59.117 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T21:07:59.117 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T21:07:59.117 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T21:07:59.117 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T21:07:59.540 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-04-26T21:07:59.547 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-04-26T21:07:59.547 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-26T21:07:59.717 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-26T21:07:59.810 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:07:59.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:07:59.959+0000 7fbc2e21e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-26T21:07:59.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:07:59.961+0000 7fbc2e21e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-26T21:07:59.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:07:59.961+0000 7fbc2e21e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T21:07:59.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:07:59.961+0000 7fbc2e21e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T21:07:59.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:07:59.961+0000 7fbc2e21e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T21:07:59.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:07:59.965+0000 7fbc2e21e640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-26T21:08:02.930 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T21:08:03.013 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:08:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:03.136+0000 7f63e040e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-26T21:08:03.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:03.136+0000 7f63e040e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-26T21:08:03.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:03.137+0000 7f63e040e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T21:08:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:03.137+0000 7f63e040e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T21:08:03.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:03.137+0000 7f63e040e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T21:08:03.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:03.139+0000 7f63e040e640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-26T21:08:06.131 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T21:08:06.218 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:08:06.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:06.343+0000 7efe0fe2d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-26T21:08:06.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:06.343+0000 7efe0fe2d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-26T21:08:06.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:06.344+0000 7efe0fe2d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T21:08:06.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:06.344+0000 7efe0fe2d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T21:08:06.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:06.344+0000 7efe0fe2d640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T21:08:06.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:06.346+0000 7efe0fe2d640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-26T21:08:06.393 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:08:06.401 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:08:09.299 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-26T21:08:09.386 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:08:09.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:09.513+0000 7f778242e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-26T21:08:09.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:09.514+0000 7f778242e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-26T21:08:09.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:09.514+0000 7f778242e640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T21:08:09.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:09.514+0000 7f778242e640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T21:08:09.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:09.514+0000 7f778242e640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T21:08:09.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:09.516+0000 7f778242e640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-26T21:08:12.475 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-26T21:08:12.558 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-26T21:08:12.558 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:08:12.558 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-26T21:08:12.652 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:08:12.652 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-26T21:08:12.742 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:08:12.743 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-26T21:08:12.829 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:08:12.829 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-26T21:08:12.912 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:08:12.912 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-26T21:08:12.997 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:08:12.997 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-26T21:08:13.083 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:08:13.083 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-26T21:08:13.174 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:08:13.174 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-26T21:08:13.263 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:08:13.263 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-26T21:08:13.349 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:08:13.349 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-26T21:08:13.433 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:08:13.434 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-26T21:08:13.512 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:08:13.512 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-26T21:08:13.638 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:08:13.639 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-26T21:08:13.720 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:08:13.720 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-26T21:08:13.802 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:08:13.802 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-26T21:08:13.862 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:08:13.862 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-26T21:08:13.915 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:08:13.915 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-26T21:08:13.970 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:08:13.971 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-26T21:08:14.024 INFO:tasks.ceph:Adding keys to all mons... 2024-04-26T21:08:14.024 DEBUG:teuthology.orchestra.run.smithi026:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T21:08:14.031 DEBUG:teuthology.orchestra.run.smithi086:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T21:08:14.111 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr.y] 2024-04-26T21:08:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCbFyxmOq38OhAAj7hQe7iTiydqLskFEDsUeA== 2024-04-26T21:08:14.111 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2024-04-26T21:08:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCcFyxmF64cEhAAsY4vNZPKu0uJ4m3oQEeKHQ== 2024-04-26T21:08:14.111 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2024-04-26T21:08:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCcFyxmF2/mHxAA3c0MGKp8Cp7DKfxfztuzuA== 2024-04-26T21:08:14.111 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.0] 2024-04-26T21:08:14.111 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCfFyxmtYeWExAA6tT0LS6D6NiVvIclusiezQ== 2024-04-26T21:08:14.111 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.1] 2024-04-26T21:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCiFyxmYPqALhAAc3wZFz3UPv1auC3/sHsOnA== 2024-04-26T21:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.2] 2024-04-26T21:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQClFyxmbNFUOxAA6tfTcUN+QfErkcilZwuz/w== 2024-04-26T21:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.3] 2024-04-26T21:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCpFyxmYzPLGxAAJ6xcdW45JCAUyr2cy7FVxA== 2024-04-26T21:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr.x] 2024-04-26T21:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCcFyxmEmYLCBAAkTnC04uqZ14rY5bgpM9PDw== 2024-04-26T21:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2024-04-26T21:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCcFyxmogPzLRAAcr843qKaNdncjkb48AIVrw== 2024-04-26T21:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2024-04-26T21:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCcFyxm6MwoOxAAnMV9jsem6T5o6Ixu4Mu5eA== 2024-04-26T21:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.4] 2024-04-26T21:08:14.112 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCvFyxmkjJEORAARc8N9amUPWZ9WhEpUJFVow== 2024-04-26T21:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.5] 2024-04-26T21:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCzFyxmEuI4CBAAOGXURY52av9wIRssjinNXQ== 2024-04-26T21:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.6] 2024-04-26T21:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQC2Fyxm4lyNFBAAU/dYFNPNZIWF/P2KwM7WyA== 2024-04-26T21:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.7] 2024-04-26T21:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQC5FyxmkNS2HhAAvCMZaOzs4ouX7YKsztShAw== 2024-04-26T21:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-26T21:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCdFyxmV7X3ChAArBm2KV2Tw+YKfYxawc6g2A== 2024-04-26T21:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout:[client.2] 2024-04-26T21:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCdFyxms6aUExAA5rF88r3JGr9aow2iyPzkoQ== 2024-04-26T21:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout:[client.1] 2024-04-26T21:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCdFyxmHtm0HRAAow48USaeLV5a06ZQH9XRSA== 2024-04-26T21:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout:[client.3] 2024-04-26T21:08:14.113 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQCdFyxmlniDJhAAzqTQ10YbKBgioFqgCWBRmA== 2024-04-26T21:08:14.114 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr.y] 2024-04-26T21:08:14.114 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCbFyxmOq38OhAAj7hQe7iTiydqLskFEDsUeA== 2024-04-26T21:08:14.114 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.a] 2024-04-26T21:08:14.114 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCcFyxmF64cEhAAsY4vNZPKu0uJ4m3oQEeKHQ== 2024-04-26T21:08:14.114 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.c] 2024-04-26T21:08:14.114 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCcFyxmF2/mHxAA3c0MGKp8Cp7DKfxfztuzuA== 2024-04-26T21:08:14.114 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.0] 2024-04-26T21:08:14.114 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCfFyxmtYeWExAA6tT0LS6D6NiVvIclusiezQ== 2024-04-26T21:08:14.114 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.1] 2024-04-26T21:08:14.115 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCiFyxmYPqALhAAc3wZFz3UPv1auC3/sHsOnA== 2024-04-26T21:08:14.115 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.2] 2024-04-26T21:08:14.115 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQClFyxmbNFUOxAA6tfTcUN+QfErkcilZwuz/w== 2024-04-26T21:08:14.115 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.3] 2024-04-26T21:08:14.115 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCpFyxmYzPLGxAAJ6xcdW45JCAUyr2cy7FVxA== 2024-04-26T21:08:14.115 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr.x] 2024-04-26T21:08:14.115 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCcFyxmEmYLCBAAkTnC04uqZ14rY5bgpM9PDw== 2024-04-26T21:08:14.115 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.b] 2024-04-26T21:08:14.115 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCcFyxmogPzLRAAcr843qKaNdncjkb48AIVrw== 2024-04-26T21:08:14.115 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.d] 2024-04-26T21:08:14.115 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCcFyxm6MwoOxAAnMV9jsem6T5o6Ixu4Mu5eA== 2024-04-26T21:08:14.115 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.4] 2024-04-26T21:08:14.116 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCvFyxmkjJEORAARc8N9amUPWZ9WhEpUJFVow== 2024-04-26T21:08:14.116 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.5] 2024-04-26T21:08:14.116 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCzFyxmEuI4CBAAOGXURY52av9wIRssjinNXQ== 2024-04-26T21:08:14.116 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.6] 2024-04-26T21:08:14.116 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQC2Fyxm4lyNFBAAU/dYFNPNZIWF/P2KwM7WyA== 2024-04-26T21:08:14.116 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.7] 2024-04-26T21:08:14.116 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQC5FyxmkNS2HhAAvCMZaOzs4ouX7YKsztShAw== 2024-04-26T21:08:14.116 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-04-26T21:08:14.116 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCdFyxmV7X3ChAArBm2KV2Tw+YKfYxawc6g2A== 2024-04-26T21:08:14.116 INFO:teuthology.orchestra.run.smithi026.stdout:[client.2] 2024-04-26T21:08:14.116 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCdFyxms6aUExAA5rF88r3JGr9aow2iyPzkoQ== 2024-04-26T21:08:14.116 INFO:teuthology.orchestra.run.smithi026.stdout:[client.1] 2024-04-26T21:08:14.116 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCdFyxmHtm0HRAAow48USaeLV5a06ZQH9XRSA== 2024-04-26T21:08:14.116 INFO:teuthology.orchestra.run.smithi026.stdout:[client.3] 2024-04-26T21:08:14.116 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCdFyxmlniDJhAAzqTQ10YbKBgioFqgCWBRmA== 2024-04-26T21:08:14.118 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T21:08:14.166 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T21:08:14.302 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:08:14.309 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:08:14.446 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:08:14.451 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:08:14.586 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:08:14.591 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:08:14.725 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:08:14.730 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:08:14.866 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:08:14.872 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:08:15.003 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:08:15.008 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:08:15.133 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T21:08:15.138 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T21:08:15.275 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:08:15.280 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:08:15.420 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:08:15.427 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T21:08:15.568 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:08:15.573 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:08:15.713 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:08:15.718 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:08:15.853 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:08:15.859 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:08:15.991 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:08:15.996 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T21:08:16.131 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:08:16.138 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:08:16.276 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:08:16.282 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:08:16.419 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:08:16.427 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:08:16.571 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:08:16.579 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T21:08:16.716 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-26T21:08:16.716 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-26T21:08:16.797 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T21:08:16.943 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-26T21:08:17.026 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-26T21:08:17.119 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T21:08:17.260 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-26T21:08:17.342 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-26T21:08:17.425 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T21:08:17.564 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-26T21:08:17.639 DEBUG:teuthology.orchestra.run.smithi026:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:08:17.647 DEBUG:teuthology.orchestra.run.smithi086:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T21:08:17.699 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-26T21:08:17.700 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-26T21:08:17.700 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-26T21:08:17.705 INFO:tasks.ceph.mon.a:Started 2024-04-26T21:08:17.705 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-26T21:08:17.705 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-26T21:08:17.710 INFO:tasks.ceph.mon.b:Started 2024-04-26T21:08:17.710 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-26T21:08:17.710 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-26T21:08:17.715 INFO:tasks.ceph.mon.c:Started 2024-04-26T21:08:17.715 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-26T21:08:17.716 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-26T21:08:17.716 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-26T21:08:17.720 INFO:tasks.ceph.mgr.y:Started 2024-04-26T21:08:17.721 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-26T21:08:17.721 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-26T21:08:17.726 INFO:tasks.ceph.mgr.x:Started 2024-04-26T21:08:17.726 INFO:tasks.ceph:Setting crush tunables to default 2024-04-26T21:08:17.726 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --cluster ceph osd crush tunables default 2024-04-26T21:08:17.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:17.897+0000 7f77de200640 1 Processor -- start 2024-04-26T21:08:17.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:17.898+0000 7f77de200640 1 -- start start 2024-04-26T21:08:17.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:17.899+0000 7f77de200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f77d815f680 0x7f77d80d5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:17.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:17.899+0000 7f77de200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f77d8160500 0x7f77d80d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:17.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:17.899+0000 7f77de200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77d80d6110 con 0x7f77d80d1b80 2024-04-26T21:08:17.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:17.899+0000 7f77de200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f77d80d0870 con 0x7f77d8160500 2024-04-26T21:08:17.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:17.899+0000 7f77de200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f77d80d1240 con 0x7f77d815f680 2024-04-26T21:08:17.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:17.899+0000 7f77cfe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f77d8160500 0x7f77d80d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:17.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:17.899+0000 7f77d7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f77d815f680 0x7f77d80d5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:17.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:17.899+0000 7f77d7e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f77d80d1b80 0x7f77d80d2d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50490/0 (socket says 172.21.15.26:50490) 2024-04-26T21:08:17.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:17.899+0000 7f77d7e00640 1 -- 172.21.15.26:0/3610091053 learned_addr learned my addr 172.21.15.26:0/3610091053 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:17.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:17.899+0000 7f77cfe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f77d8160500 0x7f77d80d0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55670/0 (socket says 172.21.15.26:55670) 2024-04-26T21:08:17.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:17.899+0000 7f77cfe00640 1 -- 172.21.15.26:0/3610091053 >> v2:172.21.15.26:3300/0 conn(0x7f77d8160500 msgr2=0x7f77d80d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T21:08:17.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:17.899+0000 7f77cfe00640 1 -- 172.21.15.26:0/3610091053 >> v2:172.21.15.26:3300/0 conn(0x7f77d8160500 msgr2=0x7f77d80d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:08:17.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:17.899+0000 7f77cfe00640 1 --2- 172.21.15.26:0/3610091053 >> v2:172.21.15.26:3300/0 conn(0x7f77d8160500 0x7f77d80d0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:08:17.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:17.899+0000 7f77cfe00640 1 --2- 172.21.15.26:0/3610091053 >> v2:172.21.15.26:3300/0 conn(0x7f77d8160500 0x7f77d80d0d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:08:17.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:17.900+0000 7f77d7400640 1 -- 172.21.15.26:0/3610091053 >> v2:172.21.15.86:3300/0 conn(0x7f77d815f680 msgr2=0x7f77d80d5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T21:08:17.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:17.900+0000 7f77d7400640 1 -- 172.21.15.26:0/3610091053 >> v2:172.21.15.86:3300/0 conn(0x7f77d815f680 msgr2=0x7f77d80d5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:08:17.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:17.900+0000 7f77d7400640 1 --2- 172.21.15.26:0/3610091053 >> v2:172.21.15.86:3300/0 conn(0x7f77d815f680 0x7f77d80d5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:08:17.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:17.900+0000 7f77d7400640 1 --2- 172.21.15.26:0/3610091053 >> v2:172.21.15.86:3300/0 conn(0x7f77d815f680 0x7f77d80d5ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:08:18.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.100+0000 7f77cfe00640 1 --2- 172.21.15.26:0/3610091053 >> v2:172.21.15.26:3300/0 conn(0x7f77d8160500 0x7f77d80d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:18.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.101+0000 7f77cfe00640 1 -- 172.21.15.26:0/3610091053 >> v2:172.21.15.26:3300/0 conn(0x7f77d8160500 msgr2=0x7f77d80d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T21:08:18.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.101+0000 7f77cfe00640 1 -- 172.21.15.26:0/3610091053 >> v2:172.21.15.26:3300/0 conn(0x7f77d8160500 msgr2=0x7f77d80d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:08:18.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.101+0000 7f77cfe00640 1 --2- 172.21.15.26:0/3610091053 >> v2:172.21.15.26:3300/0 conn(0x7f77d8160500 0x7f77d80d0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:08:18.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.101+0000 7f77cfe00640 1 --2- 172.21.15.26:0/3610091053 >> v2:172.21.15.26:3300/0 conn(0x7f77d8160500 0x7f77d80d0d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:08:18.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.101+0000 7f77d7400640 1 --2- 172.21.15.26:0/3610091053 >> v2:172.21.15.86:3300/0 conn(0x7f77d815f680 0x7f77d80d5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:18.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.102+0000 7f77d7400640 1 -- 172.21.15.26:0/3610091053 >> v2:172.21.15.86:3300/0 conn(0x7f77d815f680 msgr2=0x7f77d80d5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T21:08:18.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.102+0000 7f77d7400640 1 -- 172.21.15.26:0/3610091053 >> v2:172.21.15.86:3300/0 conn(0x7f77d815f680 msgr2=0x7f77d80d5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T21:08:18.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.102+0000 7f77d7400640 1 --2- 172.21.15.26:0/3610091053 >> v2:172.21.15.86:3300/0 conn(0x7f77d815f680 0x7f77d80d5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:08:18.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.102+0000 7f77d7400640 1 --2- 172.21.15.26:0/3610091053 >> v2:172.21.15.86:3300/0 conn(0x7f77d815f680 0x7f77d80d5ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:08:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.152+0000 7f77d6a00640 1 -- 172.21.15.26:0/3610091053 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 779372244 0 0) 0x7f77d80d6110 con 0x7f77d80d1b80 2024-04-26T21:08:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.152+0000 7f77d6a00640 1 -- 172.21.15.26:0/3610091053 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77bc003520 con 0x7f77d80d1b80 2024-04-26T21:08:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.153+0000 7f77d6a00640 1 -- 172.21.15.26:0/3610091053 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 144776029 0 0) 0x7f77bc003520 con 0x7f77d80d1b80 2024-04-26T21:08:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.153+0000 7f77d6a00640 1 -- 172.21.15.26:0/3610091053 >> v2:172.21.15.86:3300/0 conn(0x7f77d815f680 msgr2=0x7f77d80d5ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:08:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.153+0000 7f77d6a00640 1 --2- 172.21.15.26:0/3610091053 >> v2:172.21.15.86:3300/0 conn(0x7f77d815f680 0x7f77d80d5ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.153+0000 7f77d6a00640 1 -- 172.21.15.26:0/3610091053 >> v2:172.21.15.26:3300/0 conn(0x7f77d8160500 msgr2=0x7f77d80d0d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:08:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.153+0000 7f77d6a00640 1 --2- 172.21.15.26:0/3610091053 >> v2:172.21.15.26:3300/0 conn(0x7f77d8160500 0x7f77d80d0d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.153+0000 7f77d6a00640 1 -- 172.21.15.26:0/3610091053 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d80cff40 con 0x7f77d80d1b80 2024-04-26T21:08:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.154+0000 7f77d6a00640 1 -- 172.21.15.26:0/3610091053 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f77d00019f0 con 0x7f77d80d1b80 2024-04-26T21:08:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.154+0000 7f77d6a00640 1 -- 172.21.15.26:0/3610091053 >> v1:172.21.15.26:6789/0 conn(0x7f77d80d1b80 legacy=0x7f77d80d2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.154+0000 7f77d6a00640 1 --2- 172.21.15.26:0/3610091053 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f77bc003a00 0x7f77bc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:18.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.154+0000 7f77d6a00640 1 --2- 172.21.15.26:0/3610091053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc004800 0x7f77bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:18.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.154+0000 7f77d6a00640 1 --2- 172.21.15.26:0/3610091053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77bc0051c0 0x7f77bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:18.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.154+0000 7f77d6a00640 1 -- 172.21.15.26:0/3610091053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f77d80d0870 con 0x7f77bc004800 2024-04-26T21:08:18.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.154+0000 7f77d6a00640 1 -- 172.21.15.26:0/3610091053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77d80d1240 con 0x7f77bc0051c0 2024-04-26T21:08:18.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.154+0000 7f77d6a00640 1 -- 172.21.15.26:0/3610091053 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f77bc00c110 con 0x7f77bc003a00 2024-04-26T21:08:18.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.155+0000 7f77cfe00640 1 --2- 172.21.15.26:0/3610091053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc004800 0x7f77bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.155+0000 7f77d7e00640 1 --2- 172.21.15.26:0/3610091053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77bc0051c0 0x7f77bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.155+0000 7f77d7400640 1 --2- 172.21.15.26:0/3610091053 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f77bc003a00 0x7f77bc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.156+0000 7f77cfe00640 1 -- 172.21.15.26:0/3610091053 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f77bc003a00 msgr2=0x7f77bc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.156+0000 7f77cfe00640 1 --2- 172.21.15.26:0/3610091053 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f77bc003a00 0x7f77bc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.156+0000 7f77cfe00640 1 -- 172.21.15.26:0/3610091053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77bc0051c0 msgr2=0x7f77bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.156+0000 7f77cfe00640 1 --2- 172.21.15.26:0/3610091053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77bc0051c0 0x7f77bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.156+0000 7f77cfe00640 1 -- 172.21.15.26:0/3610091053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77bc003520 con 0x7f77bc004800 2024-04-26T21:08:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.156+0000 7f77d7e00640 1 -- 172.21.15.26:0/3610091053 reap_dead start 2024-04-26T21:08:18.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.158+0000 7f77cfe00640 1 --2- 172.21.15.26:0/3610091053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc004800 0x7f77bc004c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77c000da90 tx=0x7f77bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e89695824a11e0f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:18.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.160+0000 7f77d6a00640 1 -- 172.21.15.26:0/3610091053 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0038070 con 0x7f77bc004800 2024-04-26T21:08:18.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.160+0000 7f77d6a00640 1 -- 172.21.15.26:0/3610091053 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c0033410 con 0x7f77bc004800 2024-04-26T21:08:18.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.160+0000 7f77d6a00640 1 -- 172.21.15.26:0/3610091053 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c000d690 con 0x7f77bc004800 2024-04-26T21:08:18.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.161+0000 7f77de200640 1 -- 172.21.15.26:0/3610091053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc004800 msgr2=0x7f77bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:18.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.161+0000 7f77de200640 1 --2- 172.21.15.26:0/3610091053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc004800 0x7f77bc004c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77c000da90 tx=0x7f77bc005830 comp rx=0 tx=0).stop 2024-04-26T21:08:18.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.162+0000 7f77de200640 1 -- 172.21.15.26:0/3610091053 shutdown_connections 2024-04-26T21:08:18.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.162+0000 7f77de200640 1 --2- 172.21.15.26:0/3610091053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc004800 0x7f77bc004c80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:18.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.162+0000 7f77de200640 1 -- 172.21.15.26:0/3610091053 >> 172.21.15.26:0/3610091053 conn(0x7f77d809e490 msgr2=0x7f77d80d51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:18.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.162+0000 7f77de200640 1 -- 172.21.15.26:0/3610091053 shutdown_connections 2024-04-26T21:08:18.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.162+0000 7f77de200640 1 -- 172.21.15.26:0/3610091053 wait complete. 2024-04-26T21:08:18.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.164+0000 7f77de200640 1 Processor -- start 2024-04-26T21:08:18.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.164+0000 7f77de200640 1 -- start start 2024-04-26T21:08:18.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.165+0000 7f77de200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77bc004800 0x7f77d80c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.165+0000 7f77de200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f77d80c9050 0x7f77d814ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.166+0000 7f77de200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77d814f240 0x7f77d8151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.166+0000 7f77de200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f77d8160940 con 0x7f77d814f240 2024-04-26T21:08:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.166+0000 7f77de200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77d80d0500 con 0x7f77bc004800 2024-04-26T21:08:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.166+0000 7f77de200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f77d80d5ae0 con 0x7f77d80c9050 2024-04-26T21:08:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.166+0000 7f77d7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77d814f240 0x7f77d8151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.166+0000 7f77d7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77d814f240 0x7f77d8151850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55730/0 (socket says 172.21.15.26:55730) 2024-04-26T21:08:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.166+0000 7f77d7e00640 1 -- 172.21.15.26:0/2074972832 learned_addr learned my addr 172.21.15.26:0/2074972832 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.166+0000 7f77cfe00640 1 --2- 172.21.15.26:0/2074972832 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f77d80c9050 0x7f77d814ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:18.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.166+0000 7f77d7400640 1 --2- 172.21.15.26:0/2074972832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77bc004800 0x7f77d80c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:18.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.167+0000 7f77d7400640 1 -- 172.21.15.26:0/2074972832 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f77d80c9050 msgr2=0x7f77d814ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:18.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.167+0000 7f77d7400640 1 --2- 172.21.15.26:0/2074972832 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f77d80c9050 0x7f77d814ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:18.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.167+0000 7f77d7400640 1 -- 172.21.15.26:0/2074972832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77d814f240 msgr2=0x7f77d8151850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:18.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.168+0000 7f77d7400640 1 --2- 172.21.15.26:0/2074972832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77d814f240 0x7f77d8151850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:18.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.168+0000 7f77d7400640 1 -- 172.21.15.26:0/2074972832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d80cb600 con 0x7f77bc004800 2024-04-26T21:08:18.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.168+0000 7f77d7e00640 1 --2- 172.21.15.26:0/2074972832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77d814f240 0x7f77d8151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:08:18.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.168+0000 7f77d7400640 1 --2- 172.21.15.26:0/2074972832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77bc004800 0x7f77d80c8b10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f77c8006fd0 tx=0x7f77c80119f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:18.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.169+0000 7f77cf400640 1 -- 172.21.15.26:0/2074972832 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c801a070 con 0x7f77bc004800 2024-04-26T21:08:18.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.169+0000 7f77cf400640 1 -- 172.21.15.26:0/2074972832 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c8004220 con 0x7f77bc004800 2024-04-26T21:08:18.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.169+0000 7f77cf400640 1 -- 172.21.15.26:0/2074972832 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8015650 con 0x7f77bc004800 2024-04-26T21:08:18.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.169+0000 7f77de200640 1 -- 172.21.15.26:0/2074972832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d80cb8e0 con 0x7f77bc004800 2024-04-26T21:08:18.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.169+0000 7f77de200640 1 -- 172.21.15.26:0/2074972832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d80cbea0 con 0x7f77bc004800 2024-04-26T21:08:18.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.169+0000 7f77cf400640 1 -- 172.21.15.26:0/2074972832 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f77c80157f0 con 0x7f77bc004800 2024-04-26T21:08:18.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.169+0000 7f77cf400640 1 -- 172.21.15.26:0/2074972832 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f77c801e400 con 0x7f77bc004800 2024-04-26T21:08:18.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.170+0000 7f77de200640 1 -- 172.21.15.26:0/2074972832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d806a700 con 0x7f77bc004800 2024-04-26T21:08:18.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.178+0000 7f77cf400640 1 -- 172.21.15.26:0/2074972832 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f77c8015cc0 con 0x7f77bc004800 2024-04-26T21:08:18.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.254+0000 7f77de200640 1 -- 172.21.15.26:0/2074972832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f77d8068770 con 0x7f77bc004800 2024-04-26T21:08:18.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.261+0000 7f77cf400640 1 -- 172.21.15.26:0/2074972832 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f77c8003610 con 0x7f77bc004800 2024-04-26T21:08:18.263 INFO:teuthology.orchestra.run.smithi026.stderr:adjusted tunables profile to default 2024-04-26T21:08:18.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.262+0000 7f77de200640 1 -- 172.21.15.26:0/2074972832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77bc004800 msgr2=0x7f77d80c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:18.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.262+0000 7f77de200640 1 --2- 172.21.15.26:0/2074972832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77bc004800 0x7f77d80c8b10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f77c8006fd0 tx=0x7f77c80119f0 comp rx=0 tx=0).stop 2024-04-26T21:08:18.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.263+0000 7f77de200640 1 -- 172.21.15.26:0/2074972832 shutdown_connections 2024-04-26T21:08:18.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.263+0000 7f77de200640 1 --2- 172.21.15.26:0/2074972832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77d814f240 0x7f77d8151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:18.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.263+0000 7f77de200640 1 --2- 172.21.15.26:0/2074972832 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f77d80c9050 0x7f77d814ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:18.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.263+0000 7f77de200640 1 --2- 172.21.15.26:0/2074972832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77bc004800 0x7f77d80c8b10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:18.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.263+0000 7f77de200640 1 -- 172.21.15.26:0/2074972832 >> 172.21.15.26:0/2074972832 conn(0x7f77d809e490 msgr2=0x7f77d815fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:18.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.263+0000 7f77de200640 1 -- 172.21.15.26:0/2074972832 shutdown_connections 2024-04-26T21:08:18.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:18.263+0000 7f77de200640 1 -- 172.21.15.26:0/2074972832 wait complete. 2024-04-26T21:08:18.285 INFO:tasks.ceph:check_enable_crimson: False 2024-04-26T21:08:18.285 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-26T21:08:18.285 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:08:18.285 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-26T21:08:18.313 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:18.311+0000 7f61f3d32040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:08:18.362 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:08:18.362 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-26T21:08:18.483 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:08:18.483 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-26T21:08:18.506 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:18.505+0000 7f61f3d32040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:08:18.556 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:08:18.556 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-26T21:08:18.575 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:18.574+0000 7f61f3d32040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:08:18.623 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:18.622+0000 7fd37e5d6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:08:18.655 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:08:18.655 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-26T21:08:18.733 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:08:18.734 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-26T21:08:18.811 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:18.810+0000 7fd37e5d6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:08:18.846 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:08:18.846 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-26T21:08:18.864 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:18.863+0000 7f61f3d32040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:08:18.877 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:18.876+0000 7fd37e5d6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:08:18.920 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:18.918+0000 7f61f3d32040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:08:18.949 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:08:18.950 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-26T21:08:19.067 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 80aba520-1a3a-455f-9941-72e6bbfdb5f9 0 2024-04-26T21:08:19.204 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:19.203+0000 7fd37e5d6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:08:19.259 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:19.259+0000 7fd37e5d6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:08:19.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.285+0000 7f793e400640 1 Processor -- start 2024-04-26T21:08:19.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.285+0000 7f793e400640 1 -- start start 2024-04-26T21:08:19.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.286+0000 7f793e400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7938162e60 0x7f79381632e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:19.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.286+0000 7f793e400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f79380b8640 0x7f79380b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:19.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.286+0000 7f793e400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79380d3870 con 0x7f793815f290 2024-04-26T21:08:19.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.287+0000 7f793e400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f79380d3b50 con 0x7f79380b8640 2024-04-26T21:08:19.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.287+0000 7f793e400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f79380d3e30 con 0x7f7938162e60 2024-04-26T21:08:19.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.287+0000 7f7937e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7938162e60 0x7f79381632e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:19.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.287+0000 7f7937e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7938162e60 0x7f79381632e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38988/0 (socket says 172.21.15.86:38988) 2024-04-26T21:08:19.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.287+0000 7f7937e00640 1 -- 172.21.15.86:0/141739834 learned_addr learned my addr 172.21.15.86:0/141739834 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-26T21:08:19.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.287+0000 7f793da00640 1 --2- 172.21.15.86:0/141739834 >> v2:172.21.15.26:3300/0 conn(0x7f79380b8640 0x7f79380b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:19.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.287+0000 7f7937e00640 1 -- 172.21.15.86:0/141739834 >> v2:172.21.15.26:3300/0 conn(0x7f79380b8640 msgr2=0x7f79380b8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:19.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.287+0000 7f7937e00640 1 --2- 172.21.15.86:0/141739834 >> v2:172.21.15.26:3300/0 conn(0x7f79380b8640 0x7f79380b8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.287+0000 7f7937e00640 1 -- 172.21.15.86:0/141739834 >> v1:172.21.15.26:6789/0 conn(0x7f793815f290 legacy=0x7f79380d5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:19.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.287+0000 7f7937e00640 1 -- 172.21.15.86:0/141739834 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79380b7e40 con 0x7f7938162e60 2024-04-26T21:08:19.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.288+0000 7f7937e00640 1 --2- 172.21.15.86:0/141739834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7938162e60 0x7f79381632e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f79380d4f50 tx=0x7f792402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=686750130bd9da2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:19.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.289+0000 7f7937400640 1 -- 172.21.15.86:0/141739834 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7924037070 con 0x7f7938162e60 2024-04-26T21:08:19.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.289+0000 7f7937400640 1 -- 172.21.15.86:0/141739834 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7924032440 con 0x7f7938162e60 2024-04-26T21:08:19.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.289+0000 7f7937400640 1 -- 172.21.15.86:0/141739834 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7924032810 con 0x7f7938162e60 2024-04-26T21:08:19.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.289+0000 7f793e400640 1 -- 172.21.15.86:0/141739834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7938162e60 msgr2=0x7f79381632e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:19.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.289+0000 7f793e400640 1 --2- 172.21.15.86:0/141739834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7938162e60 0x7f79381632e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f79380d4f50 tx=0x7f792402a270 comp rx=0 tx=0).stop 2024-04-26T21:08:19.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.289+0000 7f793e400640 1 -- 172.21.15.86:0/141739834 shutdown_connections 2024-04-26T21:08:19.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.289+0000 7f793e400640 1 --2- 172.21.15.86:0/141739834 >> v2:172.21.15.26:3300/0 conn(0x7f79380b8640 0x7f79380b8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.289+0000 7f793e400640 1 --2- 172.21.15.86:0/141739834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7938162e60 0x7f79381632e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.289+0000 7f793e400640 1 -- 172.21.15.86:0/141739834 >> 172.21.15.86:0/141739834 conn(0x7f793809e380 msgr2=0x7f79380d4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:19.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.289+0000 7f793e400640 1 -- 172.21.15.86:0/141739834 shutdown_connections 2024-04-26T21:08:19.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.289+0000 7f793e400640 1 -- 172.21.15.86:0/141739834 wait complete. 2024-04-26T21:08:19.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.291+0000 7f793e400640 1 Processor -- start 2024-04-26T21:08:19.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.291+0000 7f793e400640 1 -- start start 2024-04-26T21:08:19.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.292+0000 7f793e400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79380b8640 0x7f7938152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:19.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.292+0000 7f793e400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f793815f290 0x7f793806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:19.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.292+0000 7f793e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7938162e60 0x7f793806dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:19.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.292+0000 7f793e400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f79380d3540 con 0x7f79380b8640 2024-04-26T21:08:19.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.292+0000 7f793e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79380d5e10 con 0x7f7938162e60 2024-04-26T21:08:19.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.292+0000 7f793e400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f793815f670 con 0x7f793815f290 2024-04-26T21:08:19.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.293+0000 7f7937e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f793815f290 0x7f793806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:19.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.293+0000 7f793da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7938162e60 0x7f793806dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:19.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.293+0000 7f7937e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f793815f290 0x7f793806d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:58620/0 (socket says 172.21.15.86:58620) 2024-04-26T21:08:19.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.293+0000 7f7937e00640 1 -- 172.21.15.86:0/867172355 learned_addr learned my addr 172.21.15.86:0/867172355 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-26T21:08:19.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.293+0000 7f793d000640 1 --2- 172.21.15.86:0/867172355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79380b8640 0x7f7938152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:19.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.293+0000 7f7937e00640 1 -- 172.21.15.86:0/867172355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7938162e60 msgr2=0x7f793806dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:19.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.293+0000 7f7937e00640 1 --2- 172.21.15.86:0/867172355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7938162e60 0x7f793806dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.293+0000 7f7937e00640 1 -- 172.21.15.86:0/867172355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79380b8640 msgr2=0x7f7938152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:19.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.293+0000 7f7937e00640 1 --2- 172.21.15.86:0/867172355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79380b8640 0x7f7938152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.293+0000 7f7937e00640 1 -- 172.21.15.86:0/867172355 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79380cd4e0 con 0x7f793815f290 2024-04-26T21:08:19.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.293+0000 7f793da00640 1 --2- 172.21.15.86:0/867172355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7938162e60 0x7f793806dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:08:19.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.293+0000 7f793d000640 1 --2- 172.21.15.86:0/867172355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79380b8640 0x7f7938152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:08:19.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.293+0000 7f7937e00640 1 --2- 172.21.15.86:0/867172355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f793815f290 0x7f793806d570 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7924000c00 tx=0x7f79240046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:19.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.294+0000 7f7934c00640 1 -- 172.21.15.86:0/867172355 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7924037040 con 0x7f793815f290 2024-04-26T21:08:19.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.294+0000 7f7934c00640 1 -- 172.21.15.86:0/867172355 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7924042070 con 0x7f793815f290 2024-04-26T21:08:19.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.294+0000 7f7934c00640 1 -- 172.21.15.86:0/867172355 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7924030880 con 0x7f793815f290 2024-04-26T21:08:19.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.294+0000 7f793e400640 1 -- 172.21.15.86:0/867172355 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79380cd7c0 con 0x7f793815f290 2024-04-26T21:08:19.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.294+0000 7f793e400640 1 -- 172.21.15.86:0/867172355 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f79380cd100 con 0x7f793815f290 2024-04-26T21:08:19.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.294+0000 7f7934c00640 1 -- 172.21.15.86:0/867172355 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7924030a20 con 0x7f793815f290 2024-04-26T21:08:19.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.294+0000 7f7934c00640 1 -- 172.21.15.86:0/867172355 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f7924047bc0 con 0x7f793815f290 2024-04-26T21:08:19.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.295+0000 7f793e400640 1 -- 172.21.15.86:0/867172355 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f793806a700 con 0x7f793815f290 2024-04-26T21:08:19.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.298+0000 7f7934c00640 1 -- 172.21.15.86:0/867172355 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7924047dc0 con 0x7f793815f290 2024-04-26T21:08:19.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.377+0000 7f793e400640 1 -- 172.21.15.86:0/867172355 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd new", "uuid": "80aba520-1a3a-455f-9941-72e6bbfdb5f9", "id": 0} v 0) -- 0x7f79380cf420 con 0x7f793815f290 2024-04-26T21:08:19.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.383+0000 7f7934c00640 1 -- 172.21.15.86:0/867172355 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "80aba520-1a3a-455f-9941-72e6bbfdb5f9", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f7924047460 con 0x7f793815f290 2024-04-26T21:08:19.384 INFO:teuthology.orchestra.run.smithi086.stdout:0 2024-04-26T21:08:19.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.384+0000 7f793e400640 1 -- 172.21.15.86:0/867172355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f793815f290 msgr2=0x7f793806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:19.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.384+0000 7f793e400640 1 --2- 172.21.15.86:0/867172355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f793815f290 0x7f793806d570 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7924000c00 tx=0x7f79240046c0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.385+0000 7f793e400640 1 -- 172.21.15.86:0/867172355 shutdown_connections 2024-04-26T21:08:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.385+0000 7f793e400640 1 --2- 172.21.15.86:0/867172355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7938162e60 0x7f793806dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.385+0000 7f793e400640 1 --2- 172.21.15.86:0/867172355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f793815f290 0x7f793806d570 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.385+0000 7f793e400640 1 --2- 172.21.15.86:0/867172355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79380b8640 0x7f7938152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.385+0000 7f793e400640 1 -- 172.21.15.86:0/867172355 >> 172.21.15.86:0/867172355 conn(0x7f793809e380 msgr2=0x7f793806e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.385+0000 7f793e400640 1 -- 172.21.15.86:0/867172355 shutdown_connections 2024-04-26T21:08:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.385+0000 7f793e400640 1 -- 172.21.15.86:0/867172355 wait complete. 2024-04-26T21:08:19.402 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 3084f542-c355-437c-8db5-6164ebc4c5f6 1 2024-04-26T21:08:19.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.579+0000 7f136b200640 1 Processor -- start 2024-04-26T21:08:19.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.579+0000 7f136b200640 1 -- start start 2024-04-26T21:08:19.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.581+0000 7f136b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f136415f600 0x7f13640d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:19.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.581+0000 7f136b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13640d3710 0x7f13640d3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:19.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.581+0000 7f136b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13640d5cf0 con 0x7f13640b3fd0 2024-04-26T21:08:19.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.581+0000 7f136b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f13640d5fd0 con 0x7f136415f600 2024-04-26T21:08:19.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.581+0000 7f136b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f13640cfee0 con 0x7f13640d3710 2024-04-26T21:08:19.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.581+0000 7f1369e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f136415f600 0x7f13640d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:19.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.581+0000 7f136a800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13640d3710 0x7f13640d3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:19.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.581+0000 7f1369400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f13640b3fd0 0x7f13640d09c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49616/0 (socket says 172.21.15.86:49616) 2024-04-26T21:08:19.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.581+0000 7f1369400640 1 -- 172.21.15.86:0/2529773591 learned_addr learned my addr 172.21.15.86:0/2529773591 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-26T21:08:19.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.581+0000 7f136a800640 1 -- 172.21.15.86:0/2529773591 >> v2:172.21.15.86:3300/0 conn(0x7f136415f600 msgr2=0x7f13640d2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:19.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.581+0000 7f136a800640 1 --2- 172.21.15.86:0/2529773591 >> v2:172.21.15.86:3300/0 conn(0x7f136415f600 0x7f13640d2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.581+0000 7f136a800640 1 -- 172.21.15.86:0/2529773591 >> v1:172.21.15.86:6789/0 conn(0x7f13640b3fd0 legacy=0x7f13640d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:19.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.582+0000 7f136a800640 1 -- 172.21.15.86:0/2529773591 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13640d01c0 con 0x7f13640d3710 2024-04-26T21:08:19.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.582+0000 7f1369e00640 1 --2- 172.21.15.86:0/2529773591 >> v2:172.21.15.86:3300/0 conn(0x7f136415f600 0x7f13640d2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:08:19.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.582+0000 7f136a800640 1 --2- 172.21.15.86:0/2529773591 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13640d3710 0x7f13640d3ba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1360009d90 tx=0x7f136002a550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4079a2cee1b6475 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:19.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.583+0000 7f1368a00640 1 -- 172.21.15.86:0/2529773591 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360007740 con 0x7f13640d3710 2024-04-26T21:08:19.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.583+0000 7f1368a00640 1 -- 172.21.15.86:0/2529773591 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13600078e0 con 0x7f13640d3710 2024-04-26T21:08:19.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.583+0000 7f1368a00640 1 -- 172.21.15.86:0/2529773591 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360007cb0 con 0x7f13640d3710 2024-04-26T21:08:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.583+0000 7f136b200640 1 -- 172.21.15.86:0/2529773591 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13640d3710 msgr2=0x7f13640d3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.583+0000 7f136b200640 1 --2- 172.21.15.86:0/2529773591 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13640d3710 0x7f13640d3ba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1360009d90 tx=0x7f136002a550 comp rx=0 tx=0).stop 2024-04-26T21:08:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.583+0000 7f136b200640 1 -- 172.21.15.86:0/2529773591 shutdown_connections 2024-04-26T21:08:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.583+0000 7f136b200640 1 --2- 172.21.15.86:0/2529773591 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13640d3710 0x7f13640d3ba0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.583+0000 7f136b200640 1 --2- 172.21.15.86:0/2529773591 >> v2:172.21.15.86:3300/0 conn(0x7f136415f600 0x7f13640d2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.583+0000 7f136b200640 1 -- 172.21.15.86:0/2529773591 >> 172.21.15.86:0/2529773591 conn(0x7f13640d5130 msgr2=0x7f13640d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.583+0000 7f136b200640 1 -- 172.21.15.86:0/2529773591 shutdown_connections 2024-04-26T21:08:19.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.584+0000 7f136b200640 1 -- 172.21.15.86:0/2529773591 wait complete. 2024-04-26T21:08:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.585+0000 7f136b200640 1 Processor -- start 2024-04-26T21:08:19.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.585+0000 7f136b200640 1 -- start start 2024-04-26T21:08:19.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.586+0000 7f136b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13640b3fd0 0x7f1364158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.586+0000 7f136b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13640d3710 0x7f13641621f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.586+0000 7f136b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f136415f600 0x7f13640b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.586+0000 7f136b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f13640d2480 con 0x7f13640b3fd0 2024-04-26T21:08:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.586+0000 7f136b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13640d10d0 con 0x7f136415f600 2024-04-26T21:08:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.586+0000 7f136b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f13640b1c90 con 0x7f13640d3710 2024-04-26T21:08:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.586+0000 7f1369400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13640d3710 0x7f13641621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.587+0000 7f1369400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13640d3710 0x7f13641621f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:58640/0 (socket says 172.21.15.86:58640) 2024-04-26T21:08:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.587+0000 7f1369400640 1 -- 172.21.15.86:0/2187864837 learned_addr learned my addr 172.21.15.86:0/2187864837 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-26T21:08:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.587+0000 7f136a800640 1 --2- 172.21.15.86:0/2187864837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f136415f600 0x7f13640b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.587+0000 7f1369e00640 1 --2- 172.21.15.86:0/2187864837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13640b3fd0 0x7f1364158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.587+0000 7f1369400640 1 -- 172.21.15.86:0/2187864837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f136415f600 msgr2=0x7f13640b5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.587+0000 7f1369400640 1 --2- 172.21.15.86:0/2187864837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f136415f600 0x7f13640b5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.587+0000 7f1369400640 1 -- 172.21.15.86:0/2187864837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13640b3fd0 msgr2=0x7f1364158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.587+0000 7f1369400640 1 --2- 172.21.15.86:0/2187864837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13640b3fd0 0x7f1364158020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.587+0000 7f1369400640 1 -- 172.21.15.86:0/2187864837 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1364161150 con 0x7f13640d3710 2024-04-26T21:08:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.587+0000 7f1369400640 1 --2- 172.21.15.86:0/2187864837 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13640d3710 0x7f13641621f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f135400c7b0 tx=0x7f135400cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.587+0000 7f135e000640 1 -- 172.21.15.86:0/2187864837 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354018070 con 0x7f13640d3710 2024-04-26T21:08:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.587+0000 7f135e000640 1 -- 172.21.15.86:0/2187864837 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13540072f0 con 0x7f13640d3710 2024-04-26T21:08:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.587+0000 7f136b200640 1 -- 172.21.15.86:0/2187864837 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1364161430 con 0x7f13640d3710 2024-04-26T21:08:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.588+0000 7f135e000640 1 -- 172.21.15.86:0/2187864837 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13540047e0 con 0x7f13640d3710 2024-04-26T21:08:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.588+0000 7f136b200640 1 -- 172.21.15.86:0/2187864837 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f13641619f0 con 0x7f13640d3710 2024-04-26T21:08:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.588+0000 7f135e000640 1 -- 172.21.15.86:0/2187864837 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f135401b070 con 0x7f13640d3710 2024-04-26T21:08:19.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.588+0000 7f135e000640 1 -- 172.21.15.86:0/2187864837 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f135401d440 con 0x7f13640d3710 2024-04-26T21:08:19.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.589+0000 7f136b200640 1 -- 172.21.15.86:0/2187864837 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13640693d0 con 0x7f13640d3710 2024-04-26T21:08:19.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.591+0000 7f135e000640 1 -- 172.21.15.86:0/2187864837 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f135400a640 con 0x7f13640d3710 2024-04-26T21:08:19.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.669+0000 7f136b200640 1 -- 172.21.15.86:0/2187864837 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3084f542-c355-437c-8db5-6164ebc4c5f6", "id": 1} v 0) -- 0x7f136408e820 con 0x7f13640d3710 2024-04-26T21:08:19.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.677+0000 7f135e000640 1 -- 172.21.15.86:0/2187864837 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3084f542-c355-437c-8db5-6164ebc4c5f6", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f1354004980 con 0x7f13640d3710 2024-04-26T21:08:19.678 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-26T21:08:19.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.678+0000 7f136b200640 1 -- 172.21.15.86:0/2187864837 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13640d3710 msgr2=0x7f13641621f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:19.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.678+0000 7f136b200640 1 --2- 172.21.15.86:0/2187864837 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13640d3710 0x7f13641621f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f135400c7b0 tx=0x7f135400cc70 comp rx=0 tx=0).stop 2024-04-26T21:08:19.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.678+0000 7f136b200640 1 -- 172.21.15.86:0/2187864837 shutdown_connections 2024-04-26T21:08:19.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.678+0000 7f136b200640 1 --2- 172.21.15.86:0/2187864837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f136415f600 0x7f13640b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.678+0000 7f136b200640 1 --2- 172.21.15.86:0/2187864837 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13640d3710 0x7f13641621f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.678+0000 7f136b200640 1 --2- 172.21.15.86:0/2187864837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13640b3fd0 0x7f1364158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.678+0000 7f136b200640 1 -- 172.21.15.86:0/2187864837 >> 172.21.15.86:0/2187864837 conn(0x7f13640d5130 msgr2=0x7f13640c8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:19.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.679+0000 7f136b200640 1 -- 172.21.15.86:0/2187864837 shutdown_connections 2024-04-26T21:08:19.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.679+0000 7f136b200640 1 -- 172.21.15.86:0/2187864837 wait complete. 2024-04-26T21:08:19.697 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new edbc1aa5-f21f-4521-a569-4e90e1384395 2 2024-04-26T21:08:19.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.863+0000 7f69e7200640 1 Processor -- start 2024-04-26T21:08:19.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.863+0000 7f69e7200640 1 -- start start 2024-04-26T21:08:19.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.864+0000 7f69e7200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69e015f600 0x7f69e00d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:19.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.865+0000 7f69e7200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69e00d5cf0 con 0x7f69e00d3710 2024-04-26T21:08:19.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.865+0000 7f69e7200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69e00d5fd0 con 0x7f69e00b3fd0 2024-04-26T21:08:19.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.865+0000 7f69e7200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f69e00cfee0 con 0x7f69e015f600 2024-04-26T21:08:19.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.865+0000 7f69e5400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f69e00b3fd0 0x7f69e00d09c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49626/0 (socket says 172.21.15.86:49626) 2024-04-26T21:08:19.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.865+0000 7f69e5400640 1 -- 172.21.15.86:0/3619344135 learned_addr learned my addr 172.21.15.86:0/3619344135 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-26T21:08:19.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.865+0000 7f69e4a00640 1 -- 172.21.15.86:0/3619344135 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 15306487 0 0) 0x7f69e00d5fd0 con 0x7f69e00b3fd0 2024-04-26T21:08:19.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.865+0000 7f69e4a00640 1 -- 172.21.15.86:0/3619344135 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69c8003520 con 0x7f69e00b3fd0 2024-04-26T21:08:19.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.866+0000 7f69e5e00640 1 --2- 172.21.15.86:0/3619344135 >> v2:172.21.15.26:3300/0 conn(0x7f69e015f600 0x7f69e00d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:19.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.866+0000 7f69e4a00640 1 -- 172.21.15.86:0/3619344135 <== mon.1 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 623528192 0 0) 0x7f69c8003520 con 0x7f69e00b3fd0 2024-04-26T21:08:19.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.866+0000 7f69e4a00640 1 -- 172.21.15.86:0/3619344135 >> v2:172.21.15.26:3300/0 conn(0x7f69e015f600 msgr2=0x7f69e00d2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:19.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.866+0000 7f69e4a00640 1 --2- 172.21.15.86:0/3619344135 >> v2:172.21.15.26:3300/0 conn(0x7f69e015f600 0x7f69e00d2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.866+0000 7f69e4a00640 1 -- 172.21.15.86:0/3619344135 >> v1:172.21.15.26:6789/0 conn(0x7f69e00d3710 legacy=0x7f69e00b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:19.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.866+0000 7f69e4a00640 1 -- 172.21.15.86:0/3619344135 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e00d01c0 con 0x7f69e00b3fd0 2024-04-26T21:08:19.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.866+0000 7f69e4a00640 1 -- 172.21.15.86:0/3619344135 <== mon.1 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f69d0001670 con 0x7f69e00b3fd0 2024-04-26T21:08:19.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.866+0000 7f69e4a00640 1 -- 172.21.15.86:0/3619344135 >> v1:172.21.15.86:6789/0 conn(0x7f69e00b3fd0 legacy=0x7f69e00d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.866+0000 7f69e4a00640 1 --2- 172.21.15.86:0/3619344135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69c8003a00 0x7f69c8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.866+0000 7f69e4a00640 1 --2- 172.21.15.86:0/3619344135 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69c8004800 0x7f69c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.866+0000 7f69e4a00640 1 --2- 172.21.15.86:0/3619344135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c8005210 0x7f69c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.866+0000 7f69e4a00640 1 -- 172.21.15.86:0/3619344135 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f69e00cfee0 con 0x7f69c8003a00 2024-04-26T21:08:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.866+0000 7f69e4a00640 1 -- 172.21.15.86:0/3619344135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69c800c160 con 0x7f69c8005210 2024-04-26T21:08:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.866+0000 7f69e4a00640 1 -- 172.21.15.86:0/3619344135 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f69c800c360 con 0x7f69c8004800 2024-04-26T21:08:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.866+0000 7f69e5400640 1 --2- 172.21.15.86:0/3619344135 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69c8004800 0x7f69c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.866+0000 7f69e6800640 1 --2- 172.21.15.86:0/3619344135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c8005210 0x7f69c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.867+0000 7f69e5e00640 1 --2- 172.21.15.86:0/3619344135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69c8003a00 0x7f69c8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.867+0000 7f69e5400640 1 -- 172.21.15.86:0/3619344135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c8005210 msgr2=0x7f69c800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.867+0000 7f69e5400640 1 --2- 172.21.15.86:0/3619344135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c8005210 0x7f69c800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.867+0000 7f69e5400640 1 -- 172.21.15.86:0/3619344135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69c8003a00 msgr2=0x7f69c8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:19.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.867+0000 7f69e5400640 1 --2- 172.21.15.86:0/3619344135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69c8003a00 0x7f69c8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.867+0000 7f69e5400640 1 -- 172.21.15.86:0/3619344135 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c8003520 con 0x7f69c8004800 2024-04-26T21:08:19.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.867+0000 7f69e6800640 1 -- 172.21.15.86:0/3619344135 reap_dead start 2024-04-26T21:08:19.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.868+0000 7f69e5400640 1 --2- 172.21.15.86:0/3619344135 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69c8004800 0x7f69c8004c80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f69d000c190 tx=0x7f69c8005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13d3825e76da60ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:19.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.868+0000 7f69e4a00640 1 -- 172.21.15.86:0/3619344135 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0007c60 con 0x7f69c8004800 2024-04-26T21:08:19.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.868+0000 7f69e4a00640 1 -- 172.21.15.86:0/3619344135 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d0005ad0 con 0x7f69c8004800 2024-04-26T21:08:19.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.868+0000 7f69e4a00640 1 -- 172.21.15.86:0/3619344135 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0007070 con 0x7f69c8004800 2024-04-26T21:08:19.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.869+0000 7f69e7200640 1 -- 172.21.15.86:0/3619344135 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69c8004800 msgr2=0x7f69c8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:19.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.869+0000 7f69e7200640 1 --2- 172.21.15.86:0/3619344135 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69c8004800 0x7f69c8004c80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f69d000c190 tx=0x7f69c8005830 comp rx=0 tx=0).stop 2024-04-26T21:08:19.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.869+0000 7f69e7200640 1 -- 172.21.15.86:0/3619344135 shutdown_connections 2024-04-26T21:08:19.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.869+0000 7f69e7200640 1 --2- 172.21.15.86:0/3619344135 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69c8004800 0x7f69c8004c80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.869+0000 7f69e7200640 1 -- 172.21.15.86:0/3619344135 >> 172.21.15.86:0/3619344135 conn(0x7f69e00d5130 msgr2=0x7f69e00d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:19.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.869+0000 7f69e7200640 1 -- 172.21.15.86:0/3619344135 shutdown_connections 2024-04-26T21:08:19.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.869+0000 7f69e7200640 1 -- 172.21.15.86:0/3619344135 wait complete. 2024-04-26T21:08:19.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.871+0000 7f69e7200640 1 Processor -- start 2024-04-26T21:08:19.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.871+0000 7f69e7200640 1 -- start start 2024-04-26T21:08:19.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.872+0000 7f69e7200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69c8004800 0x7f69e015f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:19.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.872+0000 7f69e7200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69e00cc790 0x7f69e015ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:19.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.872+0000 7f69e7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69e00b6030 0x7f69e00b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:19.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.872+0000 7f69e7200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f69e00d2f10 con 0x7f69c8004800 2024-04-26T21:08:19.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.872+0000 7f69e7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69e00d2480 con 0x7f69e00b6030 2024-04-26T21:08:19.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.872+0000 7f69e7200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f69e00d3090 con 0x7f69e00cc790 2024-04-26T21:08:19.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.872+0000 7f69e5400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69e00cc790 0x7f69e015ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:19.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.872+0000 7f69e6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69e00b6030 0x7f69e00b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:19.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.872+0000 7f69e5400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69e00cc790 0x7f69e015ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:58668/0 (socket says 172.21.15.86:58668) 2024-04-26T21:08:19.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.872+0000 7f69e5400640 1 -- 172.21.15.86:0/2135807503 learned_addr learned my addr 172.21.15.86:0/2135807503 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-26T21:08:19.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.872+0000 7f69e5e00640 1 --2- 172.21.15.86:0/2135807503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69c8004800 0x7f69e015f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:19.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.873+0000 7f69e6800640 1 -- 172.21.15.86:0/2135807503 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69e00cc790 msgr2=0x7f69e015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:19.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.873+0000 7f69e6800640 1 --2- 172.21.15.86:0/2135807503 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69e00cc790 0x7f69e015ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.873+0000 7f69e6800640 1 -- 172.21.15.86:0/2135807503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69c8004800 msgr2=0x7f69e015f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:19.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.873+0000 7f69e6800640 1 --2- 172.21.15.86:0/2135807503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69c8004800 0x7f69e015f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.873+0000 7f69e6800640 1 -- 172.21.15.86:0/2135807503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e00ccd00 con 0x7f69e00b6030 2024-04-26T21:08:19.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.873+0000 7f69e5e00640 1 --2- 172.21.15.86:0/2135807503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69c8004800 0x7f69e015f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:08:19.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.873+0000 7f69e6800640 1 --2- 172.21.15.86:0/2135807503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69e00b6030 0x7f69e00b64b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f69e008e230 tx=0x7f69dc00fd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:19.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.873+0000 7f69da000640 1 -- 172.21.15.86:0/2135807503 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc003b80 con 0x7f69e00b6030 2024-04-26T21:08:19.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.873+0000 7f69da000640 1 -- 172.21.15.86:0/2135807503 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69dc018070 con 0x7f69e00b6030 2024-04-26T21:08:19.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.873+0000 7f69e7200640 1 -- 172.21.15.86:0/2135807503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e00ccfe0 con 0x7f69e00b6030 2024-04-26T21:08:19.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.873+0000 7f69da000640 1 -- 172.21.15.86:0/2135807503 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc0050d0 con 0x7f69e00b6030 2024-04-26T21:08:19.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.874+0000 7f69e7200640 1 -- 172.21.15.86:0/2135807503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e00cd5a0 con 0x7f69e00b6030 2024-04-26T21:08:19.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.874+0000 7f69da000640 1 -- 172.21.15.86:0/2135807503 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f69dc005270 con 0x7f69e00b6030 2024-04-26T21:08:19.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.874+0000 7f69da000640 1 -- 172.21.15.86:0/2135807503 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f69dc00dda0 con 0x7f69e00b6030 2024-04-26T21:08:19.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.875+0000 7f69e7200640 1 -- 172.21.15.86:0/2135807503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e00693d0 con 0x7f69e00b6030 2024-04-26T21:08:19.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.878+0000 7f69da000640 1 -- 172.21.15.86:0/2135807503 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f69dc005560 con 0x7f69e00b6030 2024-04-26T21:08:19.921 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:19.919+0000 7f61f3d32040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:08:19.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.954+0000 7f69e7200640 1 -- 172.21.15.86:0/2135807503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd new", "uuid": "edbc1aa5-f21f-4521-a569-4e90e1384395", "id": 2} v 0) -- 0x7f69e00d34f0 con 0x7f69e00b6030 2024-04-26T21:08:19.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.961+0000 7f69da000640 1 -- 172.21.15.86:0/2135807503 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "edbc1aa5-f21f-4521-a569-4e90e1384395", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f69dc00d620 con 0x7f69e00b6030 2024-04-26T21:08:19.961 INFO:teuthology.orchestra.run.smithi086.stdout:2 2024-04-26T21:08:19.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.962+0000 7f69e7200640 1 -- 172.21.15.86:0/2135807503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69e00b6030 msgr2=0x7f69e00b64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:19.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.962+0000 7f69e7200640 1 --2- 172.21.15.86:0/2135807503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69e00b6030 0x7f69e00b64b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f69e008e230 tx=0x7f69dc00fd40 comp rx=0 tx=0).stop 2024-04-26T21:08:19.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.962+0000 7f69e7200640 1 -- 172.21.15.86:0/2135807503 shutdown_connections 2024-04-26T21:08:19.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.962+0000 7f69e7200640 1 --2- 172.21.15.86:0/2135807503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69e00b6030 0x7f69e00b64b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.962+0000 7f69e7200640 1 --2- 172.21.15.86:0/2135807503 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69e00cc790 0x7f69e015ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.962+0000 7f69e7200640 1 --2- 172.21.15.86:0/2135807503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69c8004800 0x7f69e015f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:19.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.962+0000 7f69e7200640 1 -- 172.21.15.86:0/2135807503 >> 172.21.15.86:0/2135807503 conn(0x7f69e00d5130 msgr2=0x7f69e015eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:19.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.962+0000 7f69e7200640 1 -- 172.21.15.86:0/2135807503 shutdown_connections 2024-04-26T21:08:19.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:19.962+0000 7f69e7200640 1 -- 172.21.15.86:0/2135807503 wait complete. 2024-04-26T21:08:19.979 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 2bbd1d2f-aeb3-42cb-bf45-041def845223 3 2024-04-26T21:08:20.028 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:20.027+0000 7f61f3d32040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:08:20.081 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:20.079+0000 7f61f3d32040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:08:20.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.155+0000 7fb779a00640 1 Processor -- start 2024-04-26T21:08:20.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.155+0000 7fb779a00640 1 -- start start 2024-04-26T21:08:20.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.157+0000 7fb779a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb77406c950 0x7fb7740b6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:20.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.157+0000 7fb779a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb774161900 0x7fb7740b6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:20.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.157+0000 7fb779a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7740b6ec0 con 0x7fb774146b90 2024-04-26T21:08:20.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.157+0000 7fb779a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fb7740b6c80 con 0x7fb774161900 2024-04-26T21:08:20.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.157+0000 7fb779a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb7740b71a0 con 0x7fb77406c950 2024-04-26T21:08:20.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.157+0000 7fb76be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb77406c950 0x7fb7740b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:20.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.157+0000 7fb76be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb77406c950 0x7fb7740b6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39050/0 (socket says 172.21.15.86:39050) 2024-04-26T21:08:20.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.157+0000 7fb76be00640 1 -- 172.21.15.86:0/484744049 learned_addr learned my addr 172.21.15.86:0/484744049 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-26T21:08:20.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.158+0000 7fb773400640 1 --2- 172.21.15.86:0/484744049 >> v2:172.21.15.26:3300/0 conn(0x7fb774161900 0x7fb7740b6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:20.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.158+0000 7fb76be00640 1 -- 172.21.15.86:0/484744049 >> v2:172.21.15.26:3300/0 conn(0x7fb774161900 msgr2=0x7fb7740b6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:20.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.158+0000 7fb76be00640 1 --2- 172.21.15.86:0/484744049 >> v2:172.21.15.26:3300/0 conn(0x7fb774161900 0x7fb7740b6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.158+0000 7fb76be00640 1 -- 172.21.15.86:0/484744049 >> v1:172.21.15.86:6789/0 conn(0x7fb774146b90 legacy=0x7fb77406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:20.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.158+0000 7fb76be00640 1 -- 172.21.15.86:0/484744049 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb774160e20 con 0x7fb77406c950 2024-04-26T21:08:20.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.159+0000 7fb76be00640 1 --2- 172.21.15.86:0/484744049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77406c950 0x7fb7740b6200 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb77406eca0 tx=0x7fb75c02a280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cec82806b25a49f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:20.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.159+0000 7fb772000640 1 -- 172.21.15.86:0/484744049 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c037070 con 0x7fb77406c950 2024-04-26T21:08:20.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.159+0000 7fb772000640 1 -- 172.21.15.86:0/484744049 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb75c032440 con 0x7fb77406c950 2024-04-26T21:08:20.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.159+0000 7fb772000640 1 -- 172.21.15.86:0/484744049 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c032810 con 0x7fb77406c950 2024-04-26T21:08:20.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.160+0000 7fb779a00640 1 -- 172.21.15.86:0/484744049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77406c950 msgr2=0x7fb7740b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:20.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.160+0000 7fb779a00640 1 --2- 172.21.15.86:0/484744049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77406c950 0x7fb7740b6200 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb77406eca0 tx=0x7fb75c02a280 comp rx=0 tx=0).stop 2024-04-26T21:08:20.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.160+0000 7fb779a00640 1 -- 172.21.15.86:0/484744049 shutdown_connections 2024-04-26T21:08:20.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.160+0000 7fb779a00640 1 --2- 172.21.15.86:0/484744049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb77406c950 0x7fb7740b6200 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.160+0000 7fb779a00640 1 --2- 172.21.15.86:0/484744049 >> v2:172.21.15.26:3300/0 conn(0x7fb774161900 0x7fb7740b6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.160+0000 7fb779a00640 1 -- 172.21.15.86:0/484744049 >> 172.21.15.86:0/484744049 conn(0x7fb7740b3be0 msgr2=0x7fb7740b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:20.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.160+0000 7fb779a00640 1 -- 172.21.15.86:0/484744049 shutdown_connections 2024-04-26T21:08:20.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.160+0000 7fb779a00640 1 -- 172.21.15.86:0/484744049 wait complete. 2024-04-26T21:08:20.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.162+0000 7fb779a00640 1 Processor -- start 2024-04-26T21:08:20.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.162+0000 7fb779a00640 1 -- start start 2024-04-26T21:08:20.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.163+0000 7fb779a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77406c950 0x7fb7740cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:20.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.163+0000 7fb779a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb774146b90 0x7fb774143050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:20.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.163+0000 7fb779a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb774161900 0x7fb774143590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:20.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.163+0000 7fb779a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb774070680 con 0x7fb77406c950 2024-04-26T21:08:20.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.163+0000 7fb779a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb7741613e0 con 0x7fb774146b90 2024-04-26T21:08:20.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.164+0000 7fb779a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb774070800 con 0x7fb774161900 2024-04-26T21:08:20.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.164+0000 7fb773400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb774161900 0x7fb774143590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:20.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.164+0000 7fb76be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb774146b90 0x7fb774143050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:20.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.164+0000 7fb773400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb774161900 0x7fb774143590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:58678/0 (socket says 172.21.15.86:58678) 2024-04-26T21:08:20.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.164+0000 7fb773400640 1 -- 172.21.15.86:0/1130268059 learned_addr learned my addr 172.21.15.86:0/1130268059 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-26T21:08:20.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.164+0000 7fb772a00640 1 --2- 172.21.15.86:0/1130268059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77406c950 0x7fb7740cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:20.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.164+0000 7fb76be00640 1 -- 172.21.15.86:0/1130268059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb774161900 msgr2=0x7fb774143590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:20.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.164+0000 7fb76be00640 1 --2- 172.21.15.86:0/1130268059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb774161900 0x7fb774143590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.164+0000 7fb76be00640 1 -- 172.21.15.86:0/1130268059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77406c950 msgr2=0x7fb7740cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:20.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.164+0000 7fb76be00640 1 --2- 172.21.15.86:0/1130268059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77406c950 0x7fb7740cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.164+0000 7fb76be00640 1 -- 172.21.15.86:0/1130268059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7741f3fd0 con 0x7fb774146b90 2024-04-26T21:08:20.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.164+0000 7fb772a00640 1 --2- 172.21.15.86:0/1130268059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77406c950 0x7fb7740cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:08:20.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.165+0000 7fb76be00640 1 --2- 172.21.15.86:0/1130268059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb774146b90 0x7fb774143050 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb75c031000 tx=0x7fb75c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:20.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.165+0000 7fb76aa00640 1 -- 172.21.15.86:0/1130268059 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c041020 con 0x7fb774146b90 2024-04-26T21:08:20.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.165+0000 7fb76aa00640 1 -- 172.21.15.86:0/1130268059 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb75c030700 con 0x7fb774146b90 2024-04-26T21:08:20.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.165+0000 7fb779a00640 1 -- 172.21.15.86:0/1130268059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7741f42b0 con 0x7fb774146b90 2024-04-26T21:08:20.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.165+0000 7fb76aa00640 1 -- 172.21.15.86:0/1130268059 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c037040 con 0x7fb774146b90 2024-04-26T21:08:20.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.166+0000 7fb779a00640 1 -- 172.21.15.86:0/1130268059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7741f4870 con 0x7fb774146b90 2024-04-26T21:08:20.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.166+0000 7fb76aa00640 1 -- 172.21.15.86:0/1130268059 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb75c044020 con 0x7fb774146b90 2024-04-26T21:08:20.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.166+0000 7fb76aa00640 1 -- 172.21.15.86:0/1130268059 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fb75c03dc40 con 0x7fb774146b90 2024-04-26T21:08:20.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.167+0000 7fb779a00640 1 -- 172.21.15.86:0/1130268059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7740693d0 con 0x7fb774146b90 2024-04-26T21:08:20.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.169+0000 7fb76aa00640 1 -- 172.21.15.86:0/1130268059 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb75c04b030 con 0x7fb774146b90 2024-04-26T21:08:20.190 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:20.190+0000 7fd37e5d6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:08:20.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.249+0000 7fb779a00640 1 -- 172.21.15.86:0/1130268059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2bbd1d2f-aeb3-42cb-bf45-041def845223", "id": 3} v 0) -- 0x7fb774140a80 con 0x7fb774146b90 2024-04-26T21:08:20.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.257+0000 7fb76aa00640 1 -- 172.21.15.86:0/1130268059 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2bbd1d2f-aeb3-42cb-bf45-041def845223", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fb75c03d420 con 0x7fb774146b90 2024-04-26T21:08:20.258 INFO:teuthology.orchestra.run.smithi086.stdout:3 2024-04-26T21:08:20.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.258+0000 7fb779a00640 1 -- 172.21.15.86:0/1130268059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb774146b90 msgr2=0x7fb774143050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:20.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.258+0000 7fb779a00640 1 --2- 172.21.15.86:0/1130268059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb774146b90 0x7fb774143050 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb75c031000 tx=0x7fb75c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.258+0000 7fb779a00640 1 -- 172.21.15.86:0/1130268059 shutdown_connections 2024-04-26T21:08:20.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.258+0000 7fb779a00640 1 --2- 172.21.15.86:0/1130268059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb774161900 0x7fb774143590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.258+0000 7fb779a00640 1 --2- 172.21.15.86:0/1130268059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb774146b90 0x7fb774143050 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.258+0000 7fb779a00640 1 --2- 172.21.15.86:0/1130268059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb77406c950 0x7fb7740cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.258+0000 7fb779a00640 1 -- 172.21.15.86:0/1130268059 >> 172.21.15.86:0/1130268059 conn(0x7fb7740b3be0 msgr2=0x7fb7740716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:20.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.258+0000 7fb779a00640 1 -- 172.21.15.86:0/1130268059 shutdown_connections 2024-04-26T21:08:20.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.258+0000 7fb779a00640 1 -- 172.21.15.86:0/1130268059 wait complete. 2024-04-26T21:08:20.275 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698 4 2024-04-26T21:08:20.281 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:20.279+0000 7f61f3d32040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:08:20.306 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:20.306+0000 7fd37e5d6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:08:20.339 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:20.337+0000 7f61f3d32040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:08:20.361 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:20.361+0000 7fd37e5d6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:08:20.402 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:20.401+0000 7f61f3d32040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:08:20.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.450+0000 7f5c87200640 1 Processor -- start 2024-04-26T21:08:20.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.450+0000 7f5c87200640 1 -- start start 2024-04-26T21:08:20.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.451+0000 7f5c87200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5c8015f600 0x7f5c800d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:20.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.451+0000 7f5c87200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5c800b3fd0 0x7f5c800d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:20.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.451+0000 7f5c87200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c800d5cf0 con 0x7f5c800d0f00 2024-04-26T21:08:20.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.452+0000 7f5c87200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f5c800d5fd0 con 0x7f5c8015f600 2024-04-26T21:08:20.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.452+0000 7f5c87200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5c800cfee0 con 0x7f5c800b3fd0 2024-04-26T21:08:20.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.452+0000 7f5c85400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5c800b3fd0 0x7f5c800d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:20.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.452+0000 7f5c85400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5c800b3fd0 0x7f5c800d09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39060/0 (socket says 172.21.15.86:39060) 2024-04-26T21:08:20.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.452+0000 7f5c85400640 1 -- 172.21.15.86:0/1916592199 learned_addr learned my addr 172.21.15.86:0/1916592199 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-26T21:08:20.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.452+0000 7f5c85e00640 1 --2- 172.21.15.86:0/1916592199 >> v2:172.21.15.26:3300/0 conn(0x7f5c8015f600 0x7f5c800d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:20.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.452+0000 7f5c85400640 1 -- 172.21.15.86:0/1916592199 >> v2:172.21.15.26:3300/0 conn(0x7f5c8015f600 msgr2=0x7f5c800d2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:20.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.452+0000 7f5c85400640 1 --2- 172.21.15.86:0/1916592199 >> v2:172.21.15.26:3300/0 conn(0x7f5c8015f600 0x7f5c800d2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.452+0000 7f5c85400640 1 -- 172.21.15.86:0/1916592199 >> v1:172.21.15.26:6789/0 conn(0x7f5c800d0f00 legacy=0x7f5c800d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:20.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.452+0000 7f5c85400640 1 -- 172.21.15.86:0/1916592199 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c800d01c0 con 0x7f5c800b3fd0 2024-04-26T21:08:20.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.453+0000 7f5c85400640 1 --2- 172.21.15.86:0/1916592199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c800b3fd0 0x7f5c800d09c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5c800d13c0 tx=0x7f5c7002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16857cd451772bae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:20.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.453+0000 7f5c84a00640 1 -- 172.21.15.86:0/1916592199 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70037070 con 0x7f5c800b3fd0 2024-04-26T21:08:20.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.454+0000 7f5c84a00640 1 -- 172.21.15.86:0/1916592199 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c70032440 con 0x7f5c800b3fd0 2024-04-26T21:08:20.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.454+0000 7f5c84a00640 1 -- 172.21.15.86:0/1916592199 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70032810 con 0x7f5c800b3fd0 2024-04-26T21:08:20.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.454+0000 7f5c87200640 1 -- 172.21.15.86:0/1916592199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c800b3fd0 msgr2=0x7f5c800d09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:20.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.454+0000 7f5c87200640 1 --2- 172.21.15.86:0/1916592199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c800b3fd0 0x7f5c800d09c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5c800d13c0 tx=0x7f5c7002a230 comp rx=0 tx=0).stop 2024-04-26T21:08:20.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.454+0000 7f5c87200640 1 -- 172.21.15.86:0/1916592199 shutdown_connections 2024-04-26T21:08:20.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.454+0000 7f5c87200640 1 --2- 172.21.15.86:0/1916592199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c800b3fd0 0x7f5c800d09c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.454+0000 7f5c87200640 1 --2- 172.21.15.86:0/1916592199 >> v2:172.21.15.26:3300/0 conn(0x7f5c8015f600 0x7f5c800d2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.454+0000 7f5c87200640 1 -- 172.21.15.86:0/1916592199 >> 172.21.15.86:0/1916592199 conn(0x7f5c800d5130 msgr2=0x7f5c800d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:20.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.454+0000 7f5c87200640 1 -- 172.21.15.86:0/1916592199 shutdown_connections 2024-04-26T21:08:20.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.454+0000 7f5c87200640 1 -- 172.21.15.86:0/1916592199 wait complete. 2024-04-26T21:08:20.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.456+0000 7f5c87200640 1 Processor -- start 2024-04-26T21:08:20.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.456+0000 7f5c87200640 1 -- start start 2024-04-26T21:08:20.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.457+0000 7f5c87200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5c800b3fd0 0x7f5c800ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:20.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.457+0000 7f5c87200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c800d0f00 0x7f5c8015bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:20.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.457+0000 7f5c87200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c8015f600 0x7f5c8015c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.457+0000 7f5c87200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5c800d3e20 con 0x7f5c8015f600 2024-04-26T21:08:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.457+0000 7f5c87200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c800d2480 con 0x7f5c800d0f00 2024-04-26T21:08:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.457+0000 7f5c87200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5c800d3fa0 con 0x7f5c800b3fd0 2024-04-26T21:08:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.457+0000 7f5c85400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c800d0f00 0x7f5c8015bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.457+0000 7f5c85e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5c800b3fd0 0x7f5c800ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.457+0000 7f5c85400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c800d0f00 0x7f5c8015bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39072/0 (socket says 172.21.15.86:39072) 2024-04-26T21:08:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.457+0000 7f5c85400640 1 -- 172.21.15.86:0/1043309880 learned_addr learned my addr 172.21.15.86:0/1043309880 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-26T21:08:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.458+0000 7f5c86800640 1 --2- 172.21.15.86:0/1043309880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c8015f600 0x7f5c8015c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.458+0000 7f5c85400640 1 -- 172.21.15.86:0/1043309880 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5c800b3fd0 msgr2=0x7f5c800ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.458+0000 7f5c85400640 1 --2- 172.21.15.86:0/1043309880 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5c800b3fd0 0x7f5c800ce8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.458+0000 7f5c85400640 1 -- 172.21.15.86:0/1043309880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c8015f600 msgr2=0x7f5c8015c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.458+0000 7f5c85400640 1 --2- 172.21.15.86:0/1043309880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c8015f600 0x7f5c8015c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.458+0000 7f5c85400640 1 -- 172.21.15.86:0/1043309880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c801605c0 con 0x7f5c800d0f00 2024-04-26T21:08:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.458+0000 7f5c85400640 1 --2- 172.21.15.86:0/1043309880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c800d0f00 0x7f5c8015bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5c70000c00 tx=0x7f5c700046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.458+0000 7f5c7a000640 1 -- 172.21.15.86:0/1043309880 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70041020 con 0x7f5c800d0f00 2024-04-26T21:08:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.458+0000 7f5c7a000640 1 -- 172.21.15.86:0/1043309880 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c700306c0 con 0x7f5c800d0f00 2024-04-26T21:08:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.458+0000 7f5c7a000640 1 -- 172.21.15.86:0/1043309880 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70037040 con 0x7f5c800d0f00 2024-04-26T21:08:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.458+0000 7f5c87200640 1 -- 172.21.15.86:0/1043309880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c801608a0 con 0x7f5c800d0f00 2024-04-26T21:08:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.459+0000 7f5c87200640 1 -- 172.21.15.86:0/1043309880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c80161380 con 0x7f5c800d0f00 2024-04-26T21:08:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.459+0000 7f5c7a000640 1 -- 172.21.15.86:0/1043309880 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5c70044020 con 0x7f5c800d0f00 2024-04-26T21:08:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.459+0000 7f5c7a000640 1 -- 172.21.15.86:0/1043309880 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f5c7003dc40 con 0x7f5c800d0f00 2024-04-26T21:08:20.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.460+0000 7f5c87200640 1 -- 172.21.15.86:0/1043309880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c800693d0 con 0x7f5c800d0f00 2024-04-26T21:08:20.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.462+0000 7f5c7a000640 1 -- 172.21.15.86:0/1043309880 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f5c7004b030 con 0x7f5c800d0f00 2024-04-26T21:08:20.535 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:20.534+0000 7f61f3d32040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:08:20.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.540+0000 7f5c87200640 1 -- 172.21.15.86:0/1043309880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698", "id": 4} v 0) -- 0x7f5c800d34f0 con 0x7f5c800d0f00 2024-04-26T21:08:20.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.546+0000 7f5c7a000640 1 -- 172.21.15.86:0/1043309880 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f5c7003d420 con 0x7f5c800d0f00 2024-04-26T21:08:20.547 INFO:teuthology.orchestra.run.smithi086.stdout:4 2024-04-26T21:08:20.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.547+0000 7f5c87200640 1 -- 172.21.15.86:0/1043309880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c800d0f00 msgr2=0x7f5c8015bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:20.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.547+0000 7f5c87200640 1 --2- 172.21.15.86:0/1043309880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c800d0f00 0x7f5c8015bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5c70000c00 tx=0x7f5c700046c0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.548+0000 7f5c87200640 1 -- 172.21.15.86:0/1043309880 shutdown_connections 2024-04-26T21:08:20.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.548+0000 7f5c87200640 1 --2- 172.21.15.86:0/1043309880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c8015f600 0x7f5c8015c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.548+0000 7f5c87200640 1 --2- 172.21.15.86:0/1043309880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c800d0f00 0x7f5c8015bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.548+0000 7f5c87200640 1 --2- 172.21.15.86:0/1043309880 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5c800b3fd0 0x7f5c800ce8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.548+0000 7f5c87200640 1 -- 172.21.15.86:0/1043309880 >> 172.21.15.86:0/1043309880 conn(0x7f5c800d5130 msgr2=0x7f5c8015e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:20.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.548+0000 7f5c87200640 1 -- 172.21.15.86:0/1043309880 shutdown_connections 2024-04-26T21:08:20.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.548+0000 7f5c87200640 1 -- 172.21.15.86:0/1043309880 wait complete. 2024-04-26T21:08:20.563 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:20.563+0000 7fd37e5d6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:08:20.565 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 6320cdc6-43d3-444f-95dc-a3ddc69cf718 5 2024-04-26T21:08:20.628 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:20.628+0000 7fd37e5d6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:08:20.693 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:20.692+0000 7fd37e5d6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:08:20.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.790+0000 7fde9ae00640 1 Processor -- start 2024-04-26T21:08:20.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.790+0000 7fde9ae00640 1 -- start start 2024-04-26T21:08:20.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.792+0000 7fde9ae00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fde9415f600 0x7fde940d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:20.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.792+0000 7fde9ae00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fde940b3fd0 0x7fde940d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:20.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.792+0000 7fde9ae00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde940d5cf0 con 0x7fde940d0f00 2024-04-26T21:08:20.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.792+0000 7fde9ae00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fde940d5fd0 con 0x7fde940b3fd0 2024-04-26T21:08:20.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.792+0000 7fde9ae00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fde940cfee0 con 0x7fde9415f600 2024-04-26T21:08:20.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.792+0000 7fde99a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fde9415f600 0x7fde940d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:20.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.792+0000 7fde99a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fde9415f600 0x7fde940d2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39088/0 (socket says 172.21.15.86:39088) 2024-04-26T21:08:20.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.792+0000 7fde99a00640 1 -- 172.21.15.86:0/135263743 learned_addr learned my addr 172.21.15.86:0/135263743 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-26T21:08:20.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.792+0000 7fde99000640 1 --2- 172.21.15.86:0/135263743 >> v2:172.21.15.26:3300/0 conn(0x7fde940b3fd0 0x7fde940d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:20.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.792+0000 7fde99a00640 1 -- 172.21.15.86:0/135263743 >> v2:172.21.15.26:3300/0 conn(0x7fde940b3fd0 msgr2=0x7fde940d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:20.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.792+0000 7fde99a00640 1 --2- 172.21.15.86:0/135263743 >> v2:172.21.15.26:3300/0 conn(0x7fde940b3fd0 0x7fde940d09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.793+0000 7fde99a00640 1 -- 172.21.15.86:0/135263743 >> v1:172.21.15.86:6789/0 conn(0x7fde940d0f00 legacy=0x7fde940d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:20.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.793+0000 7fde99a00640 1 -- 172.21.15.86:0/135263743 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde940d01c0 con 0x7fde9415f600 2024-04-26T21:08:20.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.793+0000 7fde93e00640 1 -- 172.21.15.86:0/135263743 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 483931638 0 0) 0x7fde940d5cf0 con 0x7fde940d0f00 2024-04-26T21:08:20.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.794+0000 7fde99a00640 1 --2- 172.21.15.86:0/135263743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9415f600 0x7fde940d2f10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fde940b7c30 tx=0x7fde8802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92ec479860f21dd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:20.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.794+0000 7fde93e00640 1 -- 172.21.15.86:0/135263743 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88037070 con 0x7fde9415f600 2024-04-26T21:08:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.794+0000 7fde93e00640 1 -- 172.21.15.86:0/135263743 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde88032440 con 0x7fde9415f600 2024-04-26T21:08:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.794+0000 7fde93e00640 1 -- 172.21.15.86:0/135263743 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88032810 con 0x7fde9415f600 2024-04-26T21:08:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.794+0000 7fde9ae00640 1 -- 172.21.15.86:0/135263743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9415f600 msgr2=0x7fde940d2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.794+0000 7fde9ae00640 1 --2- 172.21.15.86:0/135263743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9415f600 0x7fde940d2f10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fde940b7c30 tx=0x7fde8802a270 comp rx=0 tx=0).stop 2024-04-26T21:08:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.795+0000 7fde9ae00640 1 -- 172.21.15.86:0/135263743 shutdown_connections 2024-04-26T21:08:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.795+0000 7fde9ae00640 1 --2- 172.21.15.86:0/135263743 >> v2:172.21.15.26:3300/0 conn(0x7fde940b3fd0 0x7fde940d09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.795+0000 7fde9ae00640 1 --2- 172.21.15.86:0/135263743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9415f600 0x7fde940d2f10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.795+0000 7fde9ae00640 1 -- 172.21.15.86:0/135263743 >> 172.21.15.86:0/135263743 conn(0x7fde940d5130 msgr2=0x7fde940d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.795+0000 7fde9ae00640 1 -- 172.21.15.86:0/135263743 shutdown_connections 2024-04-26T21:08:20.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.795+0000 7fde9ae00640 1 -- 172.21.15.86:0/135263743 wait complete. 2024-04-26T21:08:20.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.796+0000 7fde9ae00640 1 Processor -- start 2024-04-26T21:08:20.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.797+0000 7fde9ae00640 1 -- start start 2024-04-26T21:08:20.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.797+0000 7fde9ae00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde940b3fd0 0x7fde941546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.797+0000 7fde9ae00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fde940d0f00 0x7fde940ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.797+0000 7fde9ae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9415f600 0x7fde940ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.797+0000 7fde9ae00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fde940d3e20 con 0x7fde940b3fd0 2024-04-26T21:08:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.797+0000 7fde9ae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde940d2480 con 0x7fde9415f600 2024-04-26T21:08:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.797+0000 7fde9ae00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fde940d3fa0 con 0x7fde940d0f00 2024-04-26T21:08:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.798+0000 7fde99000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fde940d0f00 0x7fde940ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.798+0000 7fde9a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9415f600 0x7fde940ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.798+0000 7fde99000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fde940d0f00 0x7fde940ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:58696/0 (socket says 172.21.15.86:58696) 2024-04-26T21:08:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.798+0000 7fde9a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9415f600 0x7fde940ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39100/0 (socket says 172.21.15.86:39100) 2024-04-26T21:08:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.798+0000 7fde99000640 1 -- 172.21.15.86:0/3020916196 learned_addr learned my addr 172.21.15.86:0/3020916196 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-26T21:08:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.798+0000 7fde99a00640 1 --2- 172.21.15.86:0/3020916196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde940b3fd0 0x7fde941546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.798+0000 7fde9a400640 1 -- 172.21.15.86:0/3020916196 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fde940d0f00 msgr2=0x7fde940ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:20.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.798+0000 7fde9a400640 1 --2- 172.21.15.86:0/3020916196 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fde940d0f00 0x7fde940ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.798+0000 7fde9a400640 1 -- 172.21.15.86:0/3020916196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde940b3fd0 msgr2=0x7fde941546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:20.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.798+0000 7fde9a400640 1 --2- 172.21.15.86:0/3020916196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde940b3fd0 0x7fde941546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.798+0000 7fde9a400640 1 -- 172.21.15.86:0/3020916196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde94154c00 con 0x7fde9415f600 2024-04-26T21:08:20.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.798+0000 7fde99a00640 1 --2- 172.21.15.86:0/3020916196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde940b3fd0 0x7fde941546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:08:20.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.798+0000 7fde9a400640 1 --2- 172.21.15.86:0/3020916196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9415f600 0x7fde940ce6b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fde940cf830 tx=0x7fde8c00cbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:20.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.799+0000 7fde91600640 1 -- 172.21.15.86:0/3020916196 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c003b70 con 0x7fde9415f600 2024-04-26T21:08:20.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.799+0000 7fde91600640 1 -- 172.21.15.86:0/3020916196 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde8c003d30 con 0x7fde9415f600 2024-04-26T21:08:20.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.799+0000 7fde9ae00640 1 -- 172.21.15.86:0/3020916196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde94160760 con 0x7fde9415f600 2024-04-26T21:08:20.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.799+0000 7fde91600640 1 -- 172.21.15.86:0/3020916196 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c0052b0 con 0x7fde9415f600 2024-04-26T21:08:20.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.799+0000 7fde9ae00640 1 -- 172.21.15.86:0/3020916196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde94160d20 con 0x7fde9415f600 2024-04-26T21:08:20.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.799+0000 7fde91600640 1 -- 172.21.15.86:0/3020916196 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fde8c005450 con 0x7fde9415f600 2024-04-26T21:08:20.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.799+0000 7fde91600640 1 -- 172.21.15.86:0/3020916196 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fde8c005740 con 0x7fde9415f600 2024-04-26T21:08:20.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.800+0000 7fde9ae00640 1 -- 172.21.15.86:0/3020916196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde940693d0 con 0x7fde9415f600 2024-04-26T21:08:20.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.802+0000 7fde91600640 1 -- 172.21.15.86:0/3020916196 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fde8c00a640 con 0x7fde9415f600 2024-04-26T21:08:20.825 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:20.825+0000 7fd37e5d6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:08:20.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.881+0000 7fde9ae00640 1 -- 172.21.15.86:0/3020916196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6320cdc6-43d3-444f-95dc-a3ddc69cf718", "id": 5} v 0) -- 0x7fde940b2dc0 con 0x7fde9415f600 2024-04-26T21:08:20.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.887+0000 7fde91600640 1 -- 172.21.15.86:0/3020916196 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6320cdc6-43d3-444f-95dc-a3ddc69cf718", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fde8c00a820 con 0x7fde9415f600 2024-04-26T21:08:20.888 INFO:teuthology.orchestra.run.smithi086.stdout:5 2024-04-26T21:08:20.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.889+0000 7fde9ae00640 1 -- 172.21.15.86:0/3020916196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9415f600 msgr2=0x7fde940ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:20.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.889+0000 7fde9ae00640 1 --2- 172.21.15.86:0/3020916196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9415f600 0x7fde940ce6b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fde940cf830 tx=0x7fde8c00cbe0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.889+0000 7fde9ae00640 1 -- 172.21.15.86:0/3020916196 shutdown_connections 2024-04-26T21:08:20.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.889+0000 7fde9ae00640 1 --2- 172.21.15.86:0/3020916196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde9415f600 0x7fde940ce6b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.889+0000 7fde9ae00640 1 --2- 172.21.15.86:0/3020916196 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fde940d0f00 0x7fde940ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.889+0000 7fde9ae00640 1 --2- 172.21.15.86:0/3020916196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde940b3fd0 0x7fde941546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:20.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.889+0000 7fde9ae00640 1 -- 172.21.15.86:0/3020916196 >> 172.21.15.86:0/3020916196 conn(0x7fde940d5130 msgr2=0x7fde940b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:20.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.889+0000 7fde9ae00640 1 -- 172.21.15.86:0/3020916196 shutdown_connections 2024-04-26T21:08:20.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:20.889+0000 7fde9ae00640 1 -- 172.21.15.86:0/3020916196 wait complete. 2024-04-26T21:08:20.906 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new be31e84f-e95f-4bd5-ae28-1d9703d8c09b 6 2024-04-26T21:08:21.013 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:21.011+0000 7f61f3d32040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:08:21.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.126+0000 7f234b200640 1 Processor -- start 2024-04-26T21:08:21.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.127+0000 7f234b200640 1 -- start start 2024-04-26T21:08:21.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.128+0000 7f234b200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f234c0d3e20 0x7f234c0d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:21.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.128+0000 7f234b200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f234c0d5cf0 con 0x7f234c15f600 2024-04-26T21:08:21.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.128+0000 7f234b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f234c0d5fd0 con 0x7f234c0b3fd0 2024-04-26T21:08:21.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.128+0000 7f234b200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f234c0cfee0 con 0x7f234c0d3e20 2024-04-26T21:08:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.128+0000 7f2349400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f234c0b3fd0 0x7f234c0d3710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49660/0 (socket says 172.21.15.86:49660) 2024-04-26T21:08:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.128+0000 7f2349400640 1 -- 172.21.15.86:0/232860248 learned_addr learned my addr 172.21.15.86:0/232860248 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-26T21:08:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.129+0000 7f234a800640 1 --2- 172.21.15.86:0/232860248 >> v2:172.21.15.26:3300/0 conn(0x7f234c0d3e20 0x7f234c0d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.129+0000 7f2348a00640 1 -- 172.21.15.86:0/232860248 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1726775033 0 0) 0x7f234c0d5fd0 con 0x7f234c0b3fd0 2024-04-26T21:08:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.129+0000 7f2348a00640 1 -- 172.21.15.86:0/232860248 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2330003520 con 0x7f234c0b3fd0 2024-04-26T21:08:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.129+0000 7f2348a00640 1 -- 172.21.15.86:0/232860248 <== mon.1 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1327653247 0 0) 0x7f2330003520 con 0x7f234c0b3fd0 2024-04-26T21:08:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.129+0000 7f2348a00640 1 -- 172.21.15.86:0/232860248 >> v2:172.21.15.26:3300/0 conn(0x7f234c0d3e20 msgr2=0x7f234c0d2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.129+0000 7f2348a00640 1 --2- 172.21.15.86:0/232860248 >> v2:172.21.15.26:3300/0 conn(0x7f234c0d3e20 0x7f234c0d2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.129+0000 7f2348a00640 1 -- 172.21.15.86:0/232860248 >> v1:172.21.15.26:6789/0 conn(0x7f234c15f600 legacy=0x7f234c0d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.129+0000 7f2348a00640 1 -- 172.21.15.86:0/232860248 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234c0d01c0 con 0x7f234c0b3fd0 2024-04-26T21:08:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.129+0000 7f2348a00640 1 -- 172.21.15.86:0/232860248 <== mon.1 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f2334001670 con 0x7f234c0b3fd0 2024-04-26T21:08:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.129+0000 7f2348a00640 1 -- 172.21.15.86:0/232860248 >> v1:172.21.15.86:6789/0 conn(0x7f234c0b3fd0 legacy=0x7f234c0d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.129+0000 7f2348a00640 1 --2- 172.21.15.86:0/232860248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2330003a00 0x7f2330003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.129+0000 7f2348a00640 1 --2- 172.21.15.86:0/232860248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2330004800 0x7f2330004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.129+0000 7f2348a00640 1 --2- 172.21.15.86:0/232860248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23300051c0 0x7f233000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.130+0000 7f2348a00640 1 -- 172.21.15.86:0/232860248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f234c0cfee0 con 0x7f2330004800 2024-04-26T21:08:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.130+0000 7f2348a00640 1 -- 172.21.15.86:0/232860248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f233000c0c0 con 0x7f2330003a00 2024-04-26T21:08:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.130+0000 7f2348a00640 1 -- 172.21.15.86:0/232860248 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f233000c2c0 con 0x7f23300051c0 2024-04-26T21:08:21.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.130+0000 7f234a800640 1 --2- 172.21.15.86:0/232860248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23300051c0 0x7f233000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:21.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.130+0000 7f2349e00640 1 --2- 172.21.15.86:0/232860248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2330003a00 0x7f2330003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:21.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.130+0000 7f2349400640 1 --2- 172.21.15.86:0/232860248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2330004800 0x7f2330004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:21.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.130+0000 7f234a800640 1 -- 172.21.15.86:0/232860248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2330003a00 msgr2=0x7f2330003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:21.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.130+0000 7f234a800640 1 --2- 172.21.15.86:0/232860248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2330003a00 0x7f2330003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.130+0000 7f234a800640 1 -- 172.21.15.86:0/232860248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2330004800 msgr2=0x7f2330004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:21.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.130+0000 7f234a800640 1 --2- 172.21.15.86:0/232860248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2330004800 0x7f2330004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.130+0000 7f234a800640 1 -- 172.21.15.86:0/232860248 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2330003520 con 0x7f23300051c0 2024-04-26T21:08:21.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.130+0000 7f2349400640 1 --2- 172.21.15.86:0/232860248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2330004800 0x7f2330004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:21.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.131+0000 7f234a800640 1 -- 172.21.15.86:0/232860248 reap_dead start 2024-04-26T21:08:21.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.131+0000 7f234a800640 1 --2- 172.21.15.86:0/232860248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23300051c0 0x7f233000b9c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f234c0cf830 tx=0x7f2330005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eb7b961c4fad89e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:21.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.131+0000 7f2348a00640 1 -- 172.21.15.86:0/232860248 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344036070 con 0x7f23300051c0 2024-04-26T21:08:21.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.131+0000 7f2348a00640 1 -- 172.21.15.86:0/232860248 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2344031400 con 0x7f23300051c0 2024-04-26T21:08:21.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.131+0000 7f2348a00640 1 -- 172.21.15.86:0/232860248 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344009690 con 0x7f23300051c0 2024-04-26T21:08:21.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.132+0000 7f234b200640 1 -- 172.21.15.86:0/232860248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23300051c0 msgr2=0x7f233000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:21.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.132+0000 7f234b200640 1 --2- 172.21.15.86:0/232860248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23300051c0 0x7f233000b9c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f234c0cf830 tx=0x7f2330005830 comp rx=0 tx=0).stop 2024-04-26T21:08:21.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.132+0000 7f234b200640 1 -- 172.21.15.86:0/232860248 shutdown_connections 2024-04-26T21:08:21.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.132+0000 7f234b200640 1 --2- 172.21.15.86:0/232860248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23300051c0 0x7f233000b9c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.132+0000 7f234b200640 1 -- 172.21.15.86:0/232860248 >> 172.21.15.86:0/232860248 conn(0x7f234c0d5130 msgr2=0x7f234c0d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:21.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.132+0000 7f234b200640 1 -- 172.21.15.86:0/232860248 shutdown_connections 2024-04-26T21:08:21.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.132+0000 7f234b200640 1 -- 172.21.15.86:0/232860248 wait complete. 2024-04-26T21:08:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.134+0000 7f234b200640 1 Processor -- start 2024-04-26T21:08:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.134+0000 7f234b200640 1 -- start start 2024-04-26T21:08:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.135+0000 7f234b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23300051c0 0x7f234c15e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.135+0000 7f234b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f234c0b64a0 0x7f234c153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.135+0000 7f234b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f234c15ff00 0x7f234c1603b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.135+0000 7f234b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f234c0d2f10 con 0x7f23300051c0 2024-04-26T21:08:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.135+0000 7f234b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f234c0d2480 con 0x7f234c15ff00 2024-04-26T21:08:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.135+0000 7f234b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f234c0d3090 con 0x7f234c0b64a0 2024-04-26T21:08:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.135+0000 7f2349400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f234c0b64a0 0x7f234c153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.135+0000 7f2349400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f234c0b64a0 0x7f234c153f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:58716/0 (socket says 172.21.15.86:58716) 2024-04-26T21:08:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.135+0000 7f2349400640 1 -- 172.21.15.86:0/917443863 learned_addr learned my addr 172.21.15.86:0/917443863 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-26T21:08:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.135+0000 7f234a800640 1 --2- 172.21.15.86:0/917443863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f234c15ff00 0x7f234c1603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.136+0000 7f2349e00640 1 --2- 172.21.15.86:0/917443863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23300051c0 0x7f234c15e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.136+0000 7f2349400640 1 -- 172.21.15.86:0/917443863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f234c15ff00 msgr2=0x7f234c1603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.136+0000 7f2349400640 1 --2- 172.21.15.86:0/917443863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f234c15ff00 0x7f234c1603b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.136+0000 7f2349400640 1 -- 172.21.15.86:0/917443863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23300051c0 msgr2=0x7f234c15e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.136+0000 7f2349400640 1 --2- 172.21.15.86:0/917443863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23300051c0 0x7f234c15e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.136+0000 7f2349400640 1 -- 172.21.15.86:0/917443863 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234c0b68c0 con 0x7f234c0b64a0 2024-04-26T21:08:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.136+0000 7f2349400640 1 --2- 172.21.15.86:0/917443863 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f234c0b64a0 0x7f234c153f40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f233400d7a0 tx=0x7f233400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.136+0000 7f2342000640 1 -- 172.21.15.86:0/917443863 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2334019070 con 0x7f234c0b64a0 2024-04-26T21:08:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.136+0000 7f2342000640 1 -- 172.21.15.86:0/917443863 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23340074e0 con 0x7f234c0b64a0 2024-04-26T21:08:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.136+0000 7f234b200640 1 -- 172.21.15.86:0/917443863 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f234c0b6b60 con 0x7f234c0b64a0 2024-04-26T21:08:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.136+0000 7f2342000640 1 -- 172.21.15.86:0/917443863 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2334004a00 con 0x7f234c0b64a0 2024-04-26T21:08:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.137+0000 7f234b200640 1 -- 172.21.15.86:0/917443863 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f234c0b7120 con 0x7f234c0b64a0 2024-04-26T21:08:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.137+0000 7f2342000640 1 -- 172.21.15.86:0/917443863 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2334004ba0 con 0x7f234c0b64a0 2024-04-26T21:08:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.137+0000 7f2342000640 1 -- 172.21.15.86:0/917443863 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f2334013470 con 0x7f234c0b64a0 2024-04-26T21:08:21.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.138+0000 7f234b200640 1 -- 172.21.15.86:0/917443863 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f234c0d5fd0 con 0x7f234c0b64a0 2024-04-26T21:08:21.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.140+0000 7f2342000640 1 -- 172.21.15.86:0/917443863 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f234c0d5fd0 con 0x7f234c0b64a0 2024-04-26T21:08:21.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.222+0000 7f234b200640 1 -- 172.21.15.86:0/917443863 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd new", "uuid": "be31e84f-e95f-4bd5-ae28-1d9703d8c09b", "id": 6} v 0) -- 0x7f234c0693d0 con 0x7f234c0b64a0 2024-04-26T21:08:21.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.230+0000 7f2342000640 1 -- 172.21.15.86:0/917443863 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "be31e84f-e95f-4bd5-ae28-1d9703d8c09b", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f2334022020 con 0x7f234c0b64a0 2024-04-26T21:08:21.232 INFO:teuthology.orchestra.run.smithi086.stdout:6 2024-04-26T21:08:21.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.232+0000 7f234b200640 1 -- 172.21.15.86:0/917443863 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f234c0b64a0 msgr2=0x7f234c153f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:21.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.232+0000 7f234b200640 1 --2- 172.21.15.86:0/917443863 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f234c0b64a0 0x7f234c153f40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f233400d7a0 tx=0x7f233400dc60 comp rx=0 tx=0).stop 2024-04-26T21:08:21.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.232+0000 7f234b200640 1 -- 172.21.15.86:0/917443863 shutdown_connections 2024-04-26T21:08:21.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.232+0000 7f234b200640 1 --2- 172.21.15.86:0/917443863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f234c15ff00 0x7f234c1603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.232+0000 7f234b200640 1 --2- 172.21.15.86:0/917443863 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f234c0b64a0 0x7f234c153f40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.232+0000 7f234b200640 1 --2- 172.21.15.86:0/917443863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23300051c0 0x7f234c15e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.232+0000 7f234b200640 1 -- 172.21.15.86:0/917443863 >> 172.21.15.86:0/917443863 conn(0x7f234c0d5130 msgr2=0x7f234c162d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:21.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.232+0000 7f234b200640 1 -- 172.21.15.86:0/917443863 shutdown_connections 2024-04-26T21:08:21.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.232+0000 7f234b200640 1 -- 172.21.15.86:0/917443863 wait complete. 2024-04-26T21:08:21.249 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 5e704415-8a84-4f70-92cc-e5b34395772f 7 2024-04-26T21:08:21.305 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:21.305+0000 7fd37e5d6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:08:21.321 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:21.320+0000 7f61f3d32040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:08:21.384 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:21.383+0000 7f61f3d32040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:08:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.426+0000 7fcfff200640 1 Processor -- start 2024-04-26T21:08:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.427+0000 7fcfff200640 1 -- start start 2024-04-26T21:08:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.428+0000 7fcfff200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd000162e60 0x7fd0001632e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.428+0000 7fcfff200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd0000b8640 0x7fd0000b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.428+0000 7fcfff200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0000d3870 con 0x7fd00015f290 2024-04-26T21:08:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.428+0000 7fcfff200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fd0000d3b50 con 0x7fd000162e60 2024-04-26T21:08:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.428+0000 7fcfff200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd0000d3e30 con 0x7fd0000b8640 2024-04-26T21:08:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.428+0000 7fcffe800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd0000b8640 0x7fd0000b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.428+0000 7fcffe800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd0000b8640 0x7fd0000b8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39118/0 (socket says 172.21.15.86:39118) 2024-04-26T21:08:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.428+0000 7fcffe800640 1 -- 172.21.15.86:0/3137762475 learned_addr learned my addr 172.21.15.86:0/3137762475 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-26T21:08:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.429+0000 7fcffd400640 1 --2- 172.21.15.86:0/3137762475 >> v2:172.21.15.26:3300/0 conn(0x7fd000162e60 0x7fd0001632e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.429+0000 7fcffe800640 1 -- 172.21.15.86:0/3137762475 >> v2:172.21.15.26:3300/0 conn(0x7fd000162e60 msgr2=0x7fd0001632e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.429+0000 7fcffe800640 1 --2- 172.21.15.86:0/3137762475 >> v2:172.21.15.26:3300/0 conn(0x7fd000162e60 0x7fd0001632e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.429+0000 7fcffe800640 1 -- 172.21.15.86:0/3137762475 >> v1:172.21.15.26:6789/0 conn(0x7fd00015f290 legacy=0x7fd0000d5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:21.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.429+0000 7fcffe800640 1 -- 172.21.15.86:0/3137762475 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0000b7e40 con 0x7fd0000b8640 2024-04-26T21:08:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.430+0000 7fcffe800640 1 --2- 172.21.15.86:0/3137762475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0000b8640 0x7fd0000b8a60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd00008f560 tx=0x7fcff802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f053d22adcca8a15 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.430+0000 7fcffca00640 1 -- 172.21.15.86:0/3137762475 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8036070 con 0x7fd0000b8640 2024-04-26T21:08:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.430+0000 7fcffca00640 1 -- 172.21.15.86:0/3137762475 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff8031600 con 0x7fd0000b8640 2024-04-26T21:08:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.430+0000 7fcffca00640 1 -- 172.21.15.86:0/3137762475 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff80319d0 con 0x7fd0000b8640 2024-04-26T21:08:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.431+0000 7fcfff200640 1 -- 172.21.15.86:0/3137762475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0000b8640 msgr2=0x7fd0000b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.431+0000 7fcfff200640 1 --2- 172.21.15.86:0/3137762475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0000b8640 0x7fd0000b8a60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd00008f560 tx=0x7fcff802a6f0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.431+0000 7fcfff200640 1 -- 172.21.15.86:0/3137762475 shutdown_connections 2024-04-26T21:08:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.431+0000 7fcfff200640 1 --2- 172.21.15.86:0/3137762475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0000b8640 0x7fd0000b8a60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.431+0000 7fcfff200640 1 --2- 172.21.15.86:0/3137762475 >> v2:172.21.15.26:3300/0 conn(0x7fd000162e60 0x7fd0001632e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.431+0000 7fcfff200640 1 -- 172.21.15.86:0/3137762475 >> 172.21.15.86:0/3137762475 conn(0x7fd00009e380 msgr2=0x7fd0000d4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.431+0000 7fcfff200640 1 -- 172.21.15.86:0/3137762475 shutdown_connections 2024-04-26T21:08:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.431+0000 7fcfff200640 1 -- 172.21.15.86:0/3137762475 wait complete. 2024-04-26T21:08:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.432+0000 7fcfff200640 1 Processor -- start 2024-04-26T21:08:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.433+0000 7fcfff200640 1 -- start start 2024-04-26T21:08:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.433+0000 7fcfff200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0000b8640 0x7fd000152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.433+0000 7fcfff200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd00015f290 0x7fd0000cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.433+0000 7fcfff200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd000162e60 0x7fd0001619e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.433+0000 7fcfff200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd0000d3540 con 0x7fd0000b8640 2024-04-26T21:08:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.433+0000 7fcfff200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd0000d5e10 con 0x7fd000162e60 2024-04-26T21:08:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.433+0000 7fcfff200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd00015f670 con 0x7fd00015f290 2024-04-26T21:08:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.434+0000 7fcffe800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd000162e60 0x7fd0001619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.434+0000 7fcffd400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd00015f290 0x7fd0000cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.434+0000 7fcffe800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd000162e60 0x7fd0001619e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39122/0 (socket says 172.21.15.86:39122) 2024-04-26T21:08:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.434+0000 7fcffe800640 1 -- 172.21.15.86:0/1346566589 learned_addr learned my addr 172.21.15.86:0/1346566589 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-26T21:08:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.434+0000 7fcffde00640 1 --2- 172.21.15.86:0/1346566589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0000b8640 0x7fd000152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.434+0000 7fcffe800640 1 -- 172.21.15.86:0/1346566589 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd00015f290 msgr2=0x7fd0000cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.434+0000 7fcffe800640 1 --2- 172.21.15.86:0/1346566589 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd00015f290 0x7fd0000cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.434+0000 7fcffe800640 1 -- 172.21.15.86:0/1346566589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0000b8640 msgr2=0x7fd000152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.434+0000 7fcffe800640 1 --2- 172.21.15.86:0/1346566589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0000b8640 0x7fd000152810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.434+0000 7fcffe800640 1 -- 172.21.15.86:0/1346566589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0000cea70 con 0x7fd000162e60 2024-04-26T21:08:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.434+0000 7fcffe800640 1 --2- 172.21.15.86:0/1346566589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd000162e60 0x7fd0001619e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcff8034040 tx=0x7fcff8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.435+0000 7fcff6000640 1 -- 172.21.15.86:0/1346566589 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8036050 con 0x7fd000162e60 2024-04-26T21:08:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.435+0000 7fcff6000640 1 -- 172.21.15.86:0/1346566589 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff8042070 con 0x7fd000162e60 2024-04-26T21:08:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.435+0000 7fcff6000640 1 -- 172.21.15.86:0/1346566589 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8009510 con 0x7fd000162e60 2024-04-26T21:08:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.435+0000 7fcfff200640 1 -- 172.21.15.86:0/1346566589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0000ced50 con 0x7fd000162e60 2024-04-26T21:08:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.435+0000 7fcfff200640 1 -- 172.21.15.86:0/1346566589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0000ccd20 con 0x7fd000162e60 2024-04-26T21:08:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.435+0000 7fcff6000640 1 -- 172.21.15.86:0/1346566589 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcff8007bf0 con 0x7fd000162e60 2024-04-26T21:08:21.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.435+0000 7fcff6000640 1 -- 172.21.15.86:0/1346566589 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fcff803dab0 con 0x7fd000162e60 2024-04-26T21:08:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.436+0000 7fcfff200640 1 -- 172.21.15.86:0/1346566589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd00006a700 con 0x7fd000162e60 2024-04-26T21:08:21.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.438+0000 7fcff6000640 1 -- 172.21.15.86:0/1346566589 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fcff803ddd0 con 0x7fd000162e60 2024-04-26T21:08:21.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.518+0000 7fcfff200640 1 -- 172.21.15.86:0/1346566589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5e704415-8a84-4f70-92cc-e5b34395772f", "id": 7} v 0) -- 0x7fd0000b76e0 con 0x7fd000162e60 2024-04-26T21:08:21.523 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:21.521+0000 7f61f3d32040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:08:21.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.528+0000 7fcff6000640 1 -- 172.21.15.86:0/1346566589 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5e704415-8a84-4f70-92cc-e5b34395772f", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fcff8047be0 con 0x7fd000162e60 2024-04-26T21:08:21.529 INFO:teuthology.orchestra.run.smithi086.stdout:7 2024-04-26T21:08:21.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.529+0000 7fcfff200640 1 -- 172.21.15.86:0/1346566589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd000162e60 msgr2=0x7fd0001619e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:21.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.529+0000 7fcfff200640 1 --2- 172.21.15.86:0/1346566589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd000162e60 0x7fd0001619e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcff8034040 tx=0x7fcff8004a40 comp rx=0 tx=0).stop 2024-04-26T21:08:21.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.529+0000 7fcfff200640 1 -- 172.21.15.86:0/1346566589 shutdown_connections 2024-04-26T21:08:21.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.529+0000 7fcfff200640 1 --2- 172.21.15.86:0/1346566589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd000162e60 0x7fd0001619e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.529+0000 7fcfff200640 1 --2- 172.21.15.86:0/1346566589 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd00015f290 0x7fd0000cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.529+0000 7fcfff200640 1 --2- 172.21.15.86:0/1346566589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0000b8640 0x7fd000152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.529+0000 7fcfff200640 1 -- 172.21.15.86:0/1346566589 >> 172.21.15.86:0/1346566589 conn(0x7fd00009e380 msgr2=0x7fd0000d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:21.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.529+0000 7fcfff200640 1 -- 172.21.15.86:0/1346566589 shutdown_connections 2024-04-26T21:08:21.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:08:21.530+0000 7fcfff200640 1 -- 172.21.15.86:0/1346566589 wait complete. 2024-04-26T21:08:21.545 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-26T21:08:21.545 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-26T21:08:21.553 INFO:tasks.ceph.osd.0:Started 2024-04-26T21:08:21.553 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-26T21:08:21.553 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-26T21:08:21.562 INFO:tasks.ceph.osd.1:Started 2024-04-26T21:08:21.562 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-26T21:08:21.562 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-26T21:08:21.567 INFO:tasks.ceph.osd.2:Started 2024-04-26T21:08:21.567 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-26T21:08:21.568 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-26T21:08:21.572 INFO:tasks.ceph.osd.3:Started 2024-04-26T21:08:21.572 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-26T21:08:21.572 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-26T21:08:21.579 INFO:tasks.ceph.osd.4:Started 2024-04-26T21:08:21.579 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-26T21:08:21.579 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-26T21:08:21.586 INFO:tasks.ceph.osd.5:Started 2024-04-26T21:08:21.586 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-26T21:08:21.586 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-26T21:08:21.591 INFO:tasks.ceph.osd.6:Started 2024-04-26T21:08:21.591 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-26T21:08:21.591 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-26T21:08:21.596 INFO:tasks.ceph.osd.7:Started 2024-04-26T21:08:21.597 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:08:21.621 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:21.621+0000 7fd37e5d6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:08:21.697 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:21.697+0000 7fd37e5d6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:08:21.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.798+0000 7f3f69a00640 1 Processor -- start 2024-04-26T21:08:21.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.798+0000 7f3f69a00640 1 -- start start 2024-04-26T21:08:21.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.799+0000 7f3f69a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3f6415f680 0x7f3f640d5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:21.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.799+0000 7f3f69a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f640d6110 con 0x7f3f640d1b80 2024-04-26T21:08:21.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.799+0000 7f3f69a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f640cff40 con 0x7f3f64160500 2024-04-26T21:08:21.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.799+0000 7f3f69a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f3f640d0870 con 0x7f3f6415f680 2024-04-26T21:08:21.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.800+0000 7f3f62a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3f6415f680 0x7f3f640d5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:21.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.800+0000 7f3f62a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3f6415f680 0x7f3f640d5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39298/0 (socket says 172.21.15.26:39298) 2024-04-26T21:08:21.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.800+0000 7f3f62a00640 1 -- 172.21.15.26:0/1186764412 learned_addr learned my addr 172.21.15.26:0/1186764412 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:21.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.800+0000 7f3f63400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3f640d1b80 0x7f3f640d2d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38502/0 (socket says 172.21.15.26:38502) 2024-04-26T21:08:21.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.800+0000 7f3f62a00640 1 -- 172.21.15.26:0/1186764412 >> v1:172.21.15.86:6789/0 conn(0x7f3f64160500 legacy=0x7f3f640d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:21.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.800+0000 7f3f62a00640 1 -- 172.21.15.26:0/1186764412 >> v1:172.21.15.26:6789/0 conn(0x7f3f640d1b80 legacy=0x7f3f640d2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:21.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.800+0000 7f3f62a00640 1 -- 172.21.15.26:0/1186764412 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f640d0220 con 0x7f3f6415f680 2024-04-26T21:08:21.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.801+0000 7f3f62a00640 1 --2- 172.21.15.26:0/1186764412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f6415f680 0x7f3f640d5ae0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009870 tx=0x7f3f5402a2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6407e322f3c20e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:21.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.801+0000 7f3f61600640 1 -- 172.21.15.26:0/1186764412 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54037070 con 0x7f3f6415f680 2024-04-26T21:08:21.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.801+0000 7f3f61600640 1 -- 172.21.15.26:0/1186764412 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f54032640 con 0x7f3f6415f680 2024-04-26T21:08:21.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.801+0000 7f3f61600640 1 -- 172.21.15.26:0/1186764412 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54032a10 con 0x7f3f6415f680 2024-04-26T21:08:21.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.802+0000 7f3f69a00640 1 -- 172.21.15.26:0/1186764412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f6415f680 msgr2=0x7f3f640d5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:21.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.802+0000 7f3f69a00640 1 --2- 172.21.15.26:0/1186764412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f6415f680 0x7f3f640d5ae0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009870 tx=0x7f3f5402a2f0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.802+0000 7f3f69a00640 1 -- 172.21.15.26:0/1186764412 shutdown_connections 2024-04-26T21:08:21.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.802+0000 7f3f69a00640 1 --2- 172.21.15.26:0/1186764412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f6415f680 0x7f3f640d5ae0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.802+0000 7f3f69a00640 1 -- 172.21.15.26:0/1186764412 >> 172.21.15.26:0/1186764412 conn(0x7f3f6409e490 msgr2=0x7f3f640d51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:21.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.802+0000 7f3f69a00640 1 -- 172.21.15.26:0/1186764412 shutdown_connections 2024-04-26T21:08:21.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.802+0000 7f3f69a00640 1 -- 172.21.15.26:0/1186764412 wait complete. 2024-04-26T21:08:21.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.804+0000 7f3f69a00640 1 Processor -- start 2024-04-26T21:08:21.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.804+0000 7f3f69a00640 1 -- start start 2024-04-26T21:08:21.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.805+0000 7f3f69a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f640d1b80 0x7f3f64152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:21.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.805+0000 7f3f69a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f6415f680 0x7f3f640ba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:21.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.805+0000 7f3f69a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f64160500 0x7f3f640b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:21.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.805+0000 7f3f69a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3f640d1fd0 con 0x7f3f640d1b80 2024-04-26T21:08:21.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.805+0000 7f3f69a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3f64160940 con 0x7f3f64160500 2024-04-26T21:08:21.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.805+0000 7f3f69a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3f640d2150 con 0x7f3f6415f680 2024-04-26T21:08:21.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.805+0000 7f3f62a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f640d1b80 0x7f3f64152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:21.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.805+0000 7f3f62a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f640d1b80 0x7f3f64152620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39304/0 (socket says 172.21.15.26:39304) 2024-04-26T21:08:21.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.805+0000 7f3f62a00640 1 -- 172.21.15.26:0/211336579 learned_addr learned my addr 172.21.15.26:0/211336579 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:21.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.805+0000 7f3f63400640 1 --2- 172.21.15.26:0/211336579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f64160500 0x7f3f640b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:21.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.805+0000 7f3f62000640 1 --2- 172.21.15.26:0/211336579 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f6415f680 0x7f3f640ba6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:21.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.806+0000 7f3f62a00640 1 -- 172.21.15.26:0/211336579 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f6415f680 msgr2=0x7f3f640ba6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:21.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.806+0000 7f3f62a00640 1 --2- 172.21.15.26:0/211336579 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f6415f680 0x7f3f640ba6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.806+0000 7f3f62a00640 1 -- 172.21.15.26:0/211336579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f64160500 msgr2=0x7f3f640b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:21.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.806+0000 7f3f62a00640 1 --2- 172.21.15.26:0/211336579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f64160500 0x7f3f640b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.806+0000 7f3f62a00640 1 -- 172.21.15.26:0/211336579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f641f9940 con 0x7f3f640d1b80 2024-04-26T21:08:21.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.806+0000 7f3f63400640 1 --2- 172.21.15.26:0/211336579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f64160500 0x7f3f640b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:08:21.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.806+0000 7f3f62a00640 1 --2- 172.21.15.26:0/211336579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f640d1b80 0x7f3f64152620 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3f54000c00 tx=0x7f3f540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:21.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.806+0000 7f3f5aa00640 1 -- 172.21.15.26:0/211336579 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54037040 con 0x7f3f640d1b80 2024-04-26T21:08:21.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.806+0000 7f3f5aa00640 1 -- 172.21.15.26:0/211336579 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f54042070 con 0x7f3f640d1b80 2024-04-26T21:08:21.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.806+0000 7f3f5aa00640 1 -- 172.21.15.26:0/211336579 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54030bd0 con 0x7f3f640d1b80 2024-04-26T21:08:21.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.806+0000 7f3f69a00640 1 -- 172.21.15.26:0/211336579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f64152b60 con 0x7f3f640d1b80 2024-04-26T21:08:21.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.807+0000 7f3f69a00640 1 -- 172.21.15.26:0/211336579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f641f90a0 con 0x7f3f640d1b80 2024-04-26T21:08:21.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.807+0000 7f3f5aa00640 1 -- 172.21.15.26:0/211336579 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3f54032500 con 0x7f3f640d1b80 2024-04-26T21:08:21.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.807+0000 7f3f5aa00640 1 -- 172.21.15.26:0/211336579 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f3f54030d70 con 0x7f3f640d1b80 2024-04-26T21:08:21.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.808+0000 7f3f69a00640 1 -- 172.21.15.26:0/211336579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f6406a700 con 0x7f3f640d1b80 2024-04-26T21:08:21.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.813+0000 7f3f5aa00640 1 -- 172.21.15.26:0/211336579 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3f64152b60 con 0x7f3f640d1b80 2024-04-26T21:08:21.857 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:21.856+0000 7fd37e5d6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:08:21.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.877+0000 7f3f69a00640 1 -- 172.21.15.26:0/211336579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f640d2760 con 0x7f3f640d1b80 2024-04-26T21:08:21.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.878+0000 7f3f5aa00640 1 -- 172.21.15.26:0/211336579 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f3f54047460 con 0x7f3f640d1b80 2024-04-26T21:08:21.880 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:21.880 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":10,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:08:21.519694+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:08:21.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.879+0000 7f3f69a00640 1 -- 172.21.15.26:0/211336579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f640d1b80 msgr2=0x7f3f64152620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:21.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.879+0000 7f3f69a00640 1 --2- 172.21.15.26:0/211336579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f640d1b80 0x7f3f64152620 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3f54000c00 tx=0x7f3f540046c0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.879+0000 7f3f69a00640 1 -- 172.21.15.26:0/211336579 shutdown_connections 2024-04-26T21:08:21.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.879+0000 7f3f69a00640 1 --2- 172.21.15.26:0/211336579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f64160500 0x7f3f640b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.879+0000 7f3f69a00640 1 --2- 172.21.15.26:0/211336579 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f6415f680 0x7f3f640ba6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.879+0000 7f3f69a00640 1 --2- 172.21.15.26:0/211336579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f640d1b80 0x7f3f64152620 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:21.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.879+0000 7f3f69a00640 1 -- 172.21.15.26:0/211336579 >> 172.21.15.26:0/211336579 conn(0x7f3f6409e490 msgr2=0x7f3f640b5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:21.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.880+0000 7f3f69a00640 1 -- 172.21.15.26:0/211336579 shutdown_connections 2024-04-26T21:08:21.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:21.880+0000 7f3f69a00640 1 -- 172.21.15.26:0/211336579 wait complete. 2024-04-26T21:08:21.897 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-26T21:08:21.898 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-26T21:08:21.925 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:21.923+0000 7f61f3d32040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:08:21.983 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:21.982+0000 7f61f3d32040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:08:22.174 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:22.172+0000 7f61f3d32040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:08:22.183 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:22.183+0000 7fd37e5d6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:08:22.239 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:22.238+0000 7fd37e5d6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:08:22.263 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-04-26T21:08:22.261+0000 7f82ad863640 -1 Falling back to public interface 2024-04-26T21:08:22.267 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-04-26T21:08:22.266+0000 7f4308c14640 -1 Falling back to public interface 2024-04-26T21:08:22.271 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-04-26T21:08:22.270+0000 7f1e39618640 -1 Falling back to public interface 2024-04-26T21:08:22.278 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-04-26T21:08:22.277+0000 7f8361884640 -1 Falling back to public interface 2024-04-26T21:08:22.279 INFO:tasks.ceph.osd.3.smithi026.stderr:2024-04-26T21:08:22.277+0000 7f4b6f6af640 -1 Falling back to public interface 2024-04-26T21:08:22.290 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-04-26T21:08:22.290+0000 7ff3d760e640 -1 Falling back to public interface 2024-04-26T21:08:22.295 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-04-26T21:08:22.294+0000 7f5008813640 -1 Falling back to public interface 2024-04-26T21:08:22.301 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-04-26T21:08:22.300+0000 7fca1be1b640 -1 Falling back to public interface 2024-04-26T21:08:22.349 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:22.348+0000 7f61f3d32040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:08:22.410 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:22.408+0000 7f61f3d32040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:08:22.428 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:22.427+0000 7fd37e5d6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:08:22.495 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:22.493+0000 7f61f3d32040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:08:22.550 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:22.548+0000 7f61f3d32040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:08:22.594 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:22.593+0000 7fd37e5d6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:08:22.653 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:22.653+0000 7fd37e5d6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:08:22.703 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T21:08:22.733 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:22.732+0000 7fd37e5d6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:08:22.785 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:22.784+0000 7fd37e5d6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:08:22.868 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.867+0000 7f87f4800640 1 Processor -- start 2024-04-26T21:08:22.869 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.867+0000 7f87f4800640 1 -- start start 2024-04-26T21:08:22.871 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.868+0000 7f87f4800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f87ec15d7a0 0x7f87ec06e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:22.871 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.868+0000 7f87f4800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f87ec06e760 0x7f87ec0b5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:22.871 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.868+0000 7f87f4800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87ec160dd0 con 0x7f87ec145ea0 2024-04-26T21:08:22.871 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.868+0000 7f87f4800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f87ec1610b0 con 0x7f87ec15d7a0 2024-04-26T21:08:22.871 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.868+0000 7f87f4800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f87ec161390 con 0x7f87ec06e760 2024-04-26T21:08:22.871 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.868+0000 7f87f1400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f87ec15d7a0 0x7f87ec06e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:22.871 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.868+0000 7f87f1e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f87ec145ea0 0x7f87ec146520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38510/0 (socket says 172.21.15.26:38510) 2024-04-26T21:08:22.871 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.869+0000 7f87f1e00640 1 -- 172.21.15.26:0/292747621 learned_addr learned my addr 172.21.15.26:0/292747621 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:22.871 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.869+0000 7f87f2800640 1 --2- 172.21.15.26:0/292747621 >> v2:172.21.15.86:3300/0 conn(0x7f87ec06e760 0x7f87ec0b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:22.872 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.869+0000 7f87f0a00640 1 -- 172.21.15.26:0/292747621 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 492397627 0 0) 0x7f87ec160dd0 con 0x7f87ec145ea0 2024-04-26T21:08:22.872 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.869+0000 7f87f0a00640 1 -- 172.21.15.26:0/292747621 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87d4003520 con 0x7f87ec145ea0 2024-04-26T21:08:22.872 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.869+0000 7f87f1400640 1 -- 172.21.15.26:0/292747621 >> v2:172.21.15.86:3300/0 conn(0x7f87ec06e760 msgr2=0x7f87ec0b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:22.872 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.869+0000 7f87f1400640 1 --2- 172.21.15.26:0/292747621 >> v2:172.21.15.86:3300/0 conn(0x7f87ec06e760 0x7f87ec0b5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:22.872 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.869+0000 7f87f1400640 1 -- 172.21.15.26:0/292747621 >> v1:172.21.15.26:6789/0 conn(0x7f87ec145ea0 legacy=0x7f87ec146520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:22.872 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.869+0000 7f87f1400640 1 -- 172.21.15.26:0/292747621 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ec161670 con 0x7f87ec15d7a0 2024-04-26T21:08:22.872 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.869+0000 7f87f2800640 1 --2- 172.21.15.26:0/292747621 >> v2:172.21.15.86:3300/0 conn(0x7f87ec06e760 0x7f87ec0b5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:08:22.873 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.871+0000 7f87f1400640 1 --2- 172.21.15.26:0/292747621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87ec15d7a0 0x7f87ec06e220 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f87ec074660 tx=0x7f87dc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe49bba33c5d5c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:22.873 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.871+0000 7f87f0a00640 1 -- 172.21.15.26:0/292747621 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc037070 con 0x7f87ec15d7a0 2024-04-26T21:08:22.873 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.871+0000 7f87f0a00640 1 -- 172.21.15.26:0/292747621 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87dc032440 con 0x7f87ec15d7a0 2024-04-26T21:08:22.874 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.871+0000 7f87f0a00640 1 -- 172.21.15.26:0/292747621 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc032810 con 0x7f87ec15d7a0 2024-04-26T21:08:22.874 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.872+0000 7f87f4800640 1 -- 172.21.15.26:0/292747621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87ec15d7a0 msgr2=0x7f87ec06e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:22.874 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.872+0000 7f87f4800640 1 --2- 172.21.15.26:0/292747621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87ec15d7a0 0x7f87ec06e220 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f87ec074660 tx=0x7f87dc02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:08:22.874 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.872+0000 7f87f4800640 1 -- 172.21.15.26:0/292747621 shutdown_connections 2024-04-26T21:08:22.874 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.872+0000 7f87f4800640 1 --2- 172.21.15.26:0/292747621 >> v2:172.21.15.86:3300/0 conn(0x7f87ec06e760 0x7f87ec0b5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:22.874 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.872+0000 7f87f4800640 1 --2- 172.21.15.26:0/292747621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87ec15d7a0 0x7f87ec06e220 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:22.874 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.872+0000 7f87f4800640 1 -- 172.21.15.26:0/292747621 >> 172.21.15.26:0/292747621 conn(0x7f87ec06fa60 msgr2=0x7f87ec06c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:22.874 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.872+0000 7f87f4800640 1 -- 172.21.15.26:0/292747621 shutdown_connections 2024-04-26T21:08:22.874 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.872+0000 7f87f4800640 1 -- 172.21.15.26:0/292747621 wait complete. 2024-04-26T21:08:22.875 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.874+0000 7f87f4800640 1 Processor -- start 2024-04-26T21:08:22.875 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.874+0000 7f87f4800640 1 -- start start 2024-04-26T21:08:22.877 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.875+0000 7f87f4800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87ec06e760 0x7f87ec15b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:22.877 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.875+0000 7f87f4800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87ec145ea0 0x7f87ec0af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:22.877 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.875+0000 7f87f4800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ec15d7a0 0x7f87ec0afef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:22.877 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.875+0000 7f87f4800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec146280 con 0x7f87ec06e760 2024-04-26T21:08:22.877 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.875+0000 7f87f4800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec161c30 con 0x7f87ec15d7a0 2024-04-26T21:08:22.877 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.875+0000 7f87f4800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f87ec146c30 con 0x7f87ec145ea0 2024-04-26T21:08:22.877 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.875+0000 7f87f1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87ec06e760 0x7f87ec15b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:22.877 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.875+0000 7f87f1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87ec06e760 0x7f87ec15b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39322/0 (socket says 172.21.15.26:39322) 2024-04-26T21:08:22.878 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.875+0000 7f87f1e00640 1 -- 172.21.15.26:0/3671140198 learned_addr learned my addr 172.21.15.26:0/3671140198 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:22.878 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.875+0000 7f87f1400640 1 --2- 172.21.15.26:0/3671140198 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87ec145ea0 0x7f87ec0af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:22.878 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.875+0000 7f87f2800640 1 --2- 172.21.15.26:0/3671140198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ec15d7a0 0x7f87ec0afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:22.878 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.875+0000 7f87f1e00640 1 -- 172.21.15.26:0/3671140198 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87ec145ea0 msgr2=0x7f87ec0af9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:22.878 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.875+0000 7f87f1e00640 1 --2- 172.21.15.26:0/3671140198 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87ec145ea0 0x7f87ec0af9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:22.878 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.875+0000 7f87f1e00640 1 -- 172.21.15.26:0/3671140198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ec15d7a0 msgr2=0x7f87ec0afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:22.878 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.875+0000 7f87f1e00640 1 --2- 172.21.15.26:0/3671140198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ec15d7a0 0x7f87ec0afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:22.878 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.875+0000 7f87f1e00640 1 -- 172.21.15.26:0/3671140198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ec13be40 con 0x7f87ec06e760 2024-04-26T21:08:22.878 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.876+0000 7f87f1e00640 1 --2- 172.21.15.26:0/3671140198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87ec06e760 0x7f87ec15b8c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f87e000c430 tx=0x7f87e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:22.879 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.876+0000 7f87e6000640 1 -- 172.21.15.26:0/3671140198 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0017070 con 0x7f87ec06e760 2024-04-26T21:08:22.879 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.876+0000 7f87e6000640 1 -- 172.21.15.26:0/3671140198 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87e0005ce0 con 0x7f87ec06e760 2024-04-26T21:08:22.879 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.876+0000 7f87e6000640 1 -- 172.21.15.26:0/3671140198 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e00072f0 con 0x7f87ec06e760 2024-04-26T21:08:22.879 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.876+0000 7f87f4800640 1 -- 172.21.15.26:0/3671140198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87ec13c120 con 0x7f87ec06e760 2024-04-26T21:08:22.879 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.876+0000 7f87f4800640 1 -- 172.21.15.26:0/3671140198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87ec0725b0 con 0x7f87ec06e760 2024-04-26T21:08:22.880 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.876+0000 7f87e6000640 1 -- 172.21.15.26:0/3671140198 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f87e001a070 con 0x7f87ec06e760 2024-04-26T21:08:22.880 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.877+0000 7f87e6000640 1 -- 172.21.15.26:0/3671140198 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f87e0012ad0 con 0x7f87ec06e760 2024-04-26T21:08:22.880 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.877+0000 7f87f4800640 1 -- 172.21.15.26:0/3671140198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87ec069d00 con 0x7f87ec06e760 2024-04-26T21:08:22.882 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.880+0000 7f87e6000640 1 -- 172.21.15.26:0/3671140198 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f87e0021020 con 0x7f87ec06e760 2024-04-26T21:08:22.949 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.948+0000 7f87f4800640 1 -- 172.21.15.26:0/3671140198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87ec06cfb0 con 0x7f87ec06e760 2024-04-26T21:08:22.950 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.949+0000 7f87e6000640 1 -- 172.21.15.26:0/3671140198 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f87e00075a0 con 0x7f87ec06e760 2024-04-26T21:08:22.950 INFO:teuthology.misc.health.smithi026.stdout: 2024-04-26T21:08:22.951 INFO:teuthology.misc.health.smithi026.stdout:{"epoch":10,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:08:21.519694+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:08:22.952 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.950+0000 7f87f4800640 1 -- 172.21.15.26:0/3671140198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87ec06e760 msgr2=0x7f87ec15b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:22.952 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.950+0000 7f87f4800640 1 --2- 172.21.15.26:0/3671140198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87ec06e760 0x7f87ec15b8c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f87e000c430 tx=0x7f87e000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:08:22.952 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.950+0000 7f87f4800640 1 -- 172.21.15.26:0/3671140198 shutdown_connections 2024-04-26T21:08:22.952 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.950+0000 7f87f4800640 1 --2- 172.21.15.26:0/3671140198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ec15d7a0 0x7f87ec0afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:22.952 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.950+0000 7f87f4800640 1 --2- 172.21.15.26:0/3671140198 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87ec145ea0 0x7f87ec0af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:22.952 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.950+0000 7f87f4800640 1 --2- 172.21.15.26:0/3671140198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87ec06e760 0x7f87ec15b8c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:22.952 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.950+0000 7f87f4800640 1 -- 172.21.15.26:0/3671140198 >> 172.21.15.26:0/3671140198 conn(0x7f87ec06fa60 msgr2=0x7f87ec15cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:22.952 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.950+0000 7f87f4800640 1 -- 172.21.15.26:0/3671140198 shutdown_connections 2024-04-26T21:08:22.952 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:22.950+0000 7f87f4800640 1 -- 172.21.15.26:0/3671140198 wait complete. 2024-04-26T21:08:22.965 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-26T21:08:23.121 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:23.120+0000 7f61f3d32040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:08:23.259 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:23.258+0000 7f61f3d32040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:08:23.331 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:23.331+0000 7fd37e5d6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:08:23.463 INFO:tasks.ceph.mgr.x.smithi086.stderr:2024-04-26T21:08:23.463+0000 7fd37e5d6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:08:24.296 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:24.295+0000 7f6193a00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-26T21:08:24.298 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-04-26T21:08:24.297+0000 7f6193a00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-26T21:08:25.695 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-04-26T21:08:25.695+0000 7f5008813640 -1 osd.2 0 log_to_monitors true 2024-04-26T21:08:25.716 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-04-26T21:08:25.716+0000 7ff3d760e640 -1 osd.4 0 log_to_monitors true 2024-04-26T21:08:25.862 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-04-26T21:08:25.861+0000 7f4308c14640 -1 osd.5 0 log_to_monitors true 2024-04-26T21:08:25.874 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-04-26T21:08:25.874+0000 7f82ad863640 -1 osd.1 0 log_to_monitors true 2024-04-26T21:08:25.891 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-04-26T21:08:25.891+0000 7f8361884640 -1 osd.0 0 log_to_monitors true 2024-04-26T21:08:25.902 INFO:tasks.ceph.osd.3.smithi026.stderr:2024-04-26T21:08:25.901+0000 7f4b6f6af640 -1 osd.3 0 log_to_monitors true 2024-04-26T21:08:25.919 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-04-26T21:08:25.919+0000 7f1e39618640 -1 osd.7 0 log_to_monitors true 2024-04-26T21:08:25.953 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-04-26T21:08:25.952+0000 7fca1be1b640 -1 osd.6 0 log_to_monitors true 2024-04-26T21:08:29.001 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-04-26T21:08:29.001+0000 7f835c800640 -1 osd.0 0 waiting for initial osdmap 2024-04-26T21:08:29.012 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-04-26T21:08:29.011+0000 7f8352800640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:08:29.036 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-04-26T21:08:29.035+0000 7f82a8800640 -1 osd.1 0 waiting for initial osdmap 2024-04-26T21:08:29.046 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-04-26T21:08:29.045+0000 7f829e800640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:08:29.053 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-04-26T21:08:29.053+0000 7f5003000640 -1 osd.2 0 waiting for initial osdmap 2024-04-26T21:08:29.054 INFO:tasks.ceph.osd.3.smithi026.stderr:2024-04-26T21:08:29.054+0000 7f4b6a600640 -1 osd.3 0 waiting for initial osdmap 2024-04-26T21:08:29.058 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-04-26T21:08:29.057+0000 7f4ff9000640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:08:29.064 INFO:tasks.ceph.osd.3.smithi026.stderr:2024-04-26T21:08:29.064+0000 7f4b60600640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:08:29.499 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-04-26T21:08:29.498+0000 7f4303e00640 -1 osd.5 0 waiting for initial osdmap 2024-04-26T21:08:29.505 INFO:tasks.ceph.osd.5.smithi086.stderr:2024-04-26T21:08:29.505+0000 7f42f9400640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:08:29.553 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-04-26T21:08:29.552+0000 7fca16600640 -1 osd.6 0 waiting for initial osdmap 2024-04-26T21:08:29.563 INFO:tasks.ceph.osd.6.smithi086.stderr:2024-04-26T21:08:29.563+0000 7fca0c600640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:08:29.600 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-04-26T21:08:29.600+0000 7f1e33e00640 -1 osd.7 0 waiting for initial osdmap 2024-04-26T21:08:29.609 INFO:tasks.ceph.osd.7.smithi086.stderr:2024-04-26T21:08:29.608+0000 7f1e29e00640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:08:29.626 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-04-26T21:08:29.625+0000 7ff3d1e00640 -1 osd.4 0 waiting for initial osdmap 2024-04-26T21:08:29.646 INFO:tasks.ceph.osd.4.smithi086.stderr:2024-04-26T21:08:29.646+0000 7ff3c7e00640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:08:29.771 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T21:08:29.966 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.965+0000 7fe2a7200640 1 Processor -- start 2024-04-26T21:08:29.966 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.965+0000 7fe2a7200640 1 -- start start 2024-04-26T21:08:29.969 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.967+0000 7fe2a7200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a815f370 0x7fe2a80b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:29.969 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.967+0000 7fe2a7200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe2a80b5c20 0x7fe2a8071f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:29.969 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.967+0000 7fe2a7200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe2a8072480 0x7fe2a8162cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:29.969 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.967+0000 7fe2a7200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fe2a806df10 con 0x7fe2a80b5c20 2024-04-26T21:08:29.969 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.967+0000 7fe2a7200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe2a80b4960 con 0x7fe2a8072480 2024-04-26T21:08:29.969 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.967+0000 7fe2a7200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a80b4c40 con 0x7fe2a815f370 2024-04-26T21:08:29.969 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.967+0000 7fe2a5400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe2a80b5c20 0x7fe2a8071f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:29.969 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.967+0000 7fe2a5400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe2a80b5c20 0x7fe2a8071f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42028/0 (socket says 172.21.15.26:42028) 2024-04-26T21:08:29.969 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.967+0000 7fe2a5400640 1 -- 172.21.15.26:0/2092746651 learned_addr learned my addr 172.21.15.26:0/2092746651 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:29.969 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.968+0000 7fe2a5e00640 1 --2- 172.21.15.26:0/2092746651 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a815f370 0x7fe2a80b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:29.969 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.968+0000 7fe2a6800640 1 --2- 172.21.15.26:0/2092746651 >> v2:172.21.15.86:3300/0 conn(0x7fe2a8072480 0x7fe2a8162cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:29.969 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.968+0000 7fe2a5400640 1 -- 172.21.15.26:0/2092746651 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a815f370 msgr2=0x7fe2a80b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:29.970 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.968+0000 7fe2a5400640 1 --2- 172.21.15.26:0/2092746651 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a815f370 0x7fe2a80b56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:29.970 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.968+0000 7fe2a5400640 1 -- 172.21.15.26:0/2092746651 >> v2:172.21.15.86:3300/0 conn(0x7fe2a8072480 msgr2=0x7fe2a8162cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:29.970 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.968+0000 7fe2a5400640 1 --2- 172.21.15.26:0/2092746651 >> v2:172.21.15.86:3300/0 conn(0x7fe2a8072480 0x7fe2a8162cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:29.970 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.968+0000 7fe2a5400640 1 -- 172.21.15.26:0/2092746651 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a806e180 con 0x7fe2a80b5c20 2024-04-26T21:08:29.970 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.968+0000 7fe2a6800640 1 --2- 172.21.15.26:0/2092746651 >> v2:172.21.15.86:3300/0 conn(0x7fe2a8072480 0x7fe2a8162cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:29.971 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.969+0000 7fe2a5400640 1 --2- 172.21.15.26:0/2092746651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2a80b5c20 0x7fe2a8071f40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8162c30 tx=0x7fe29402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51785d9fa4b508c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:29.971 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.969+0000 7fe2a4a00640 1 -- 172.21.15.26:0/2092746651 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294037070 con 0x7fe2a80b5c20 2024-04-26T21:08:29.971 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.969+0000 7fe2a4a00640 1 -- 172.21.15.26:0/2092746651 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe294032440 con 0x7fe2a80b5c20 2024-04-26T21:08:29.971 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.969+0000 7fe2a4a00640 1 -- 172.21.15.26:0/2092746651 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294032810 con 0x7fe2a80b5c20 2024-04-26T21:08:29.971 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.970+0000 7fe2a7200640 1 -- 172.21.15.26:0/2092746651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2a80b5c20 msgr2=0x7fe2a8071f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:29.971 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.970+0000 7fe2a7200640 1 --2- 172.21.15.26:0/2092746651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2a80b5c20 0x7fe2a8071f40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8162c30 tx=0x7fe29402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:08:29.971 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.970+0000 7fe2a7200640 1 -- 172.21.15.26:0/2092746651 shutdown_connections 2024-04-26T21:08:29.971 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.970+0000 7fe2a7200640 1 --2- 172.21.15.26:0/2092746651 >> v2:172.21.15.86:3300/0 conn(0x7fe2a8072480 0x7fe2a8162cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:29.972 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.970+0000 7fe2a7200640 1 --2- 172.21.15.26:0/2092746651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2a80b5c20 0x7fe2a8071f40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:29.972 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.970+0000 7fe2a7200640 1 --2- 172.21.15.26:0/2092746651 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a815f370 0x7fe2a80b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:29.972 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.970+0000 7fe2a7200640 1 -- 172.21.15.26:0/2092746651 >> 172.21.15.26:0/2092746651 conn(0x7fe2a80b28e0 msgr2=0x7fe2a80b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:29.972 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.970+0000 7fe2a7200640 1 -- 172.21.15.26:0/2092746651 shutdown_connections 2024-04-26T21:08:29.972 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.970+0000 7fe2a7200640 1 -- 172.21.15.26:0/2092746651 wait complete. 2024-04-26T21:08:29.972 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.972+0000 7fe2a7200640 1 Processor -- start 2024-04-26T21:08:29.972 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.972+0000 7fe2a7200640 1 -- start start 2024-04-26T21:08:29.974 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.973+0000 7fe2a7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a8072480 0x7fe2a8150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:29.974 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.973+0000 7fe2a7200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a80b5c20 0x7fe2a809d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:29.974 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.973+0000 7fe2a7200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2a8160ba0 0x7fe2a81612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:29.974 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.973+0000 7fe2a7200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a8163210 con 0x7fe2a8160ba0 2024-04-26T21:08:29.974 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.973+0000 7fe2a7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a80b45f0 con 0x7fe2a8072480 2024-04-26T21:08:29.974 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.973+0000 7fe2a7200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a8163390 con 0x7fe2a80b5c20 2024-04-26T21:08:29.974 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.973+0000 7fe2a6800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2a8160ba0 0x7fe2a81612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:29.975 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.973+0000 7fe2a6800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2a8160ba0 0x7fe2a81612e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42036/0 (socket says 172.21.15.26:42036) 2024-04-26T21:08:29.975 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.973+0000 7fe2a6800640 1 -- 172.21.15.26:0/4225817950 learned_addr learned my addr 172.21.15.26:0/4225817950 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:29.975 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.973+0000 7fe2a5e00640 1 --2- 172.21.15.26:0/4225817950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a8072480 0x7fe2a8150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:29.975 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.973+0000 7fe2a5400640 1 --2- 172.21.15.26:0/4225817950 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a80b5c20 0x7fe2a809d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:29.976 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.974+0000 7fe2a6800640 1 -- 172.21.15.26:0/4225817950 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a80b5c20 msgr2=0x7fe2a809d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:29.976 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.974+0000 7fe2a6800640 1 --2- 172.21.15.26:0/4225817950 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a80b5c20 0x7fe2a809d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:29.976 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.974+0000 7fe2a6800640 1 -- 172.21.15.26:0/4225817950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a8072480 msgr2=0x7fe2a8150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:29.976 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.974+0000 7fe2a6800640 1 --2- 172.21.15.26:0/4225817950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a8072480 0x7fe2a8150530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:29.976 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.974+0000 7fe2a6800640 1 -- 172.21.15.26:0/4225817950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a809da50 con 0x7fe2a8160ba0 2024-04-26T21:08:29.976 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.974+0000 7fe2a5e00640 1 --2- 172.21.15.26:0/4225817950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a8072480 0x7fe2a8150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:08:29.976 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.974+0000 7fe2a6800640 1 --2- 172.21.15.26:0/4225817950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2a8160ba0 0x7fe2a81612e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000cef0 tx=0x7fe2a000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:29.977 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.974+0000 7fe29e000640 1 -- 172.21.15.26:0/4225817950 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a001a070 con 0x7fe2a8160ba0 2024-04-26T21:08:29.977 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.975+0000 7fe29e000640 1 -- 172.21.15.26:0/4225817950 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a000d840 con 0x7fe2a8160ba0 2024-04-26T21:08:29.977 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.975+0000 7fe29e000640 1 -- 172.21.15.26:0/4225817950 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a000dc30 con 0x7fe2a8160ba0 2024-04-26T21:08:29.977 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.975+0000 7fe2a7200640 1 -- 172.21.15.26:0/4225817950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a809dd30 con 0x7fe2a8160ba0 2024-04-26T21:08:29.977 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.975+0000 7fe2a7200640 1 -- 172.21.15.26:0/4225817950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a809e2f0 con 0x7fe2a8160ba0 2024-04-26T21:08:29.977 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.976+0000 7fe2a7200640 1 -- 172.21.15.26:0/4225817950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a8076740 con 0x7fe2a8160ba0 2024-04-26T21:08:29.977 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.977+0000 7fe29e000640 1 -- 172.21.15.26:0/4225817950 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fe2a001d070 con 0x7fe2a8160ba0 2024-04-26T21:08:29.978 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.977+0000 7fe29e000640 1 --2- 172.21.15.26:0/4225817950 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe28806c860 0x7fe28806ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:29.978 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.977+0000 7fe29e000640 1 -- 172.21.15.26:0/4225817950 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7fe2a008b4d0 con 0x7fe2a8160ba0 2024-04-26T21:08:29.978 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.978+0000 7fe2a5e00640 1 --2- 172.21.15.26:0/4225817950 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe28806c860 0x7fe28806ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:29.979 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.978+0000 7fe2a5e00640 1 --2- 172.21.15.26:0/4225817950 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe28806c860 0x7fe28806ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80b3e20 tx=0x7fe298008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:29.983 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:29.983+0000 7fe29e000640 1 -- 172.21.15.26:0/4225817950 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe2a005a080 con 0x7fe2a8160ba0 2024-04-26T21:08:30.202 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:30.202+0000 7fe2a7200640 1 -- 172.21.15.26:0/4225817950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2a8150a70 con 0x7fe2a8160ba0 2024-04-26T21:08:30.203 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:30.203+0000 7fe29e000640 1 -- 172.21.15.26:0/4225817950 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10861 (secure 0 0 0) 0x7fe2a005e380 con 0x7fe2a8160ba0 2024-04-26T21:08:30.204 INFO:teuthology.misc.health.smithi026.stdout: 2024-04-26T21:08:30.204 INFO:teuthology.misc.health.smithi026.stdout:{"epoch":13,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:08:30.006391+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:08:30.206 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:30.205+0000 7fe2a7200640 1 -- 172.21.15.26:0/4225817950 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe28806c860 msgr2=0x7fe28806ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:30.206 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:30.205+0000 7fe2a7200640 1 --2- 172.21.15.26:0/4225817950 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe28806c860 0x7fe28806ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80b3e20 tx=0x7fe298008040 comp rx=0 tx=0).stop 2024-04-26T21:08:30.207 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:30.205+0000 7fe2a7200640 1 -- 172.21.15.26:0/4225817950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2a8160ba0 msgr2=0x7fe2a81612e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:30.207 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:30.205+0000 7fe2a7200640 1 --2- 172.21.15.26:0/4225817950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2a8160ba0 0x7fe2a81612e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000cef0 tx=0x7fe2a000a710 comp rx=0 tx=0).stop 2024-04-26T21:08:30.207 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:30.206+0000 7fe2a7200640 1 -- 172.21.15.26:0/4225817950 shutdown_connections 2024-04-26T21:08:30.207 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:30.206+0000 7fe2a7200640 1 --2- 172.21.15.26:0/4225817950 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe28806c860 0x7fe28806ed20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.207 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:30.206+0000 7fe2a7200640 1 --2- 172.21.15.26:0/4225817950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2a8160ba0 0x7fe2a81612e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.207 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:30.206+0000 7fe2a7200640 1 --2- 172.21.15.26:0/4225817950 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2a80b5c20 0x7fe2a809d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.207 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:30.206+0000 7fe2a7200640 1 --2- 172.21.15.26:0/4225817950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2a8072480 0x7fe2a8150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.207 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:30.206+0000 7fe2a7200640 1 -- 172.21.15.26:0/4225817950 >> 172.21.15.26:0/4225817950 conn(0x7fe2a80b28e0 msgr2=0x7fe2a80b5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:30.207 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:30.206+0000 7fe2a7200640 1 -- 172.21.15.26:0/4225817950 shutdown_connections 2024-04-26T21:08:30.207 INFO:teuthology.misc.health.smithi026.stderr:2024-04-26T21:08:30.206+0000 7fe2a7200640 1 -- 172.21.15.26:0/4225817950 wait complete. 2024-04-26T21:08:30.220 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-26T21:08:30.221 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-26T21:08:30.221 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:08:30.221 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:08:30.226 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:08:30.226 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:08:30.226 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:08:30.230 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:08:30.230 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:08:30.230 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:08:30.238 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:08:30.238 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:08:30.238 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:08:30.246 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:08:30.246 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-26T21:08:30.246 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:08:30.371 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-04-26T21:08:30.372 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-04-26T21:08:30.388 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2024-04-26T21:08:30.403 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2024-04-26T21:08:30.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.427+0000 7f967be00640 1 Processor -- start 2024-04-26T21:08:30.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.427+0000 7f967be00640 1 -- start start 2024-04-26T21:08:30.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.428+0000 7f967be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f967c0d1d60 0x7f967c0b7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:30.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.428+0000 7f967be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f967c0b8470 0x7f967c162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:30.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.428+0000 7f967be00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f967c0d3d80 con 0x7f967c0d1980 2024-04-26T21:08:30.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.428+0000 7f967be00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f967c0d04e0 con 0x7f967c0b8470 2024-04-26T21:08:30.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.428+0000 7f967be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f967c0cfe90 con 0x7f967c0d1d60 2024-04-26T21:08:30.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.428+0000 7f967b400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f967c0b8470 0x7f967c162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:30.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.428+0000 7f967a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f967c0d1d60 0x7f967c0b7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:30.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.428+0000 7f967a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f967c0d1d60 0x7f967c0b7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.26:60932/0 (socket says 172.21.15.26:60932) 2024-04-26T21:08:30.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.428+0000 7f967a000640 1 -- 172.21.15.26:0/1612807866 learned_addr learned my addr 172.21.15.26:0/1612807866 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:30.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.428+0000 7f967aa00640 1 --1- 172.21.15.26:0/1612807866 >> v1:172.21.15.86:6789/0 conn(0x7f967c0d1980 0x7f967c0d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.26:53976/0 (socket says 172.21.15.26:53976) 2024-04-26T21:08:30.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.429+0000 7f967a000640 1 -- 172.21.15.26:0/1612807866 >> v2:172.21.15.26:3300/0 conn(0x7f967c0b8470 msgr2=0x7f967c162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:30.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.429+0000 7f967a000640 1 --2- 172.21.15.26:0/1612807866 >> v2:172.21.15.26:3300/0 conn(0x7f967c0b8470 0x7f967c162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.429+0000 7f967a000640 1 -- 172.21.15.26:0/1612807866 >> v1:172.21.15.86:6789/0 conn(0x7f967c0d1980 legacy=0x7f967c0d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:30.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.429+0000 7f967a000640 1 -- 172.21.15.26:0/1612807866 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f967c0d0170 con 0x7f967c0d1d60 2024-04-26T21:08:30.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.430+0000 7f967a000640 1 --2- 172.21.15.26:0/1612807866 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f967c0d1d60 0x7f967c0b7f30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9668009870 tx=0x7f966802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7db39a1d3df589c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:30.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.430+0000 7f9679600640 1 -- 172.21.15.26:0/1612807866 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668037070 con 0x7f967c0d1d60 2024-04-26T21:08:30.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.430+0000 7f9679600640 1 -- 172.21.15.26:0/1612807866 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9668032440 con 0x7f967c0d1d60 2024-04-26T21:08:30.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.430+0000 7f9679600640 1 -- 172.21.15.26:0/1612807866 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668032810 con 0x7f967c0d1d60 2024-04-26T21:08:30.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.431+0000 7f967be00640 1 -- 172.21.15.26:0/1612807866 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f967c0d1d60 msgr2=0x7f967c0b7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:30.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.431+0000 7f967be00640 1 --2- 172.21.15.26:0/1612807866 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f967c0d1d60 0x7f967c0b7f30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9668009870 tx=0x7f966802a250 comp rx=0 tx=0).stop 2024-04-26T21:08:30.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.431+0000 7f967be00640 1 -- 172.21.15.26:0/1612807866 shutdown_connections 2024-04-26T21:08:30.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.431+0000 7f967be00640 1 --2- 172.21.15.26:0/1612807866 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f967c0d1d60 0x7f967c0b7f30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.431+0000 7f967be00640 1 --2- 172.21.15.26:0/1612807866 >> v2:172.21.15.26:3300/0 conn(0x7f967c0b8470 0x7f967c162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.431+0000 7f967be00640 1 -- 172.21.15.26:0/1612807866 >> 172.21.15.26:0/1612807866 conn(0x7f967c09e410 msgr2=0x7f967c0d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:30.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.431+0000 7f967be00640 1 -- 172.21.15.26:0/1612807866 shutdown_connections 2024-04-26T21:08:30.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.431+0000 7f967be00640 1 -- 172.21.15.26:0/1612807866 wait complete. 2024-04-26T21:08:30.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.432+0000 7f967be00640 1 Processor -- start 2024-04-26T21:08:30.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.432+0000 7f967be00640 1 -- start start 2024-04-26T21:08:30.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.433+0000 7f967be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f967c0b8470 0x7f967c14e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:30.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.433+0000 7f967be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f967c0d1980 0x7f967c0c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:30.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.433+0000 7f967be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f967c0d1d60 0x7f967c0c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.433+0000 7f967be00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f967c0d41e0 con 0x7f967c0d1d60 2024-04-26T21:08:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.433+0000 7f967be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f967c0d4060 con 0x7f967c0d1980 2024-04-26T21:08:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.433+0000 7f967be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f967c162ab0 con 0x7f967c0b8470 2024-04-26T21:08:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.434+0000 7f967b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f967c0d1d60 0x7f967c0c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.434+0000 7f967b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f967c0d1d60 0x7f967c0c9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42074/0 (socket says 172.21.15.26:42074) 2024-04-26T21:08:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.434+0000 7f967b400640 1 -- 172.21.15.26:0/3112965162 learned_addr learned my addr 172.21.15.26:0/3112965162 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.434+0000 7f967aa00640 1 --2- 172.21.15.26:0/3112965162 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f967c0b8470 0x7f967c14e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.434+0000 7f967a000640 1 --2- 172.21.15.26:0/3112965162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f967c0d1980 0x7f967c0c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.434+0000 7f967b400640 1 -- 172.21.15.26:0/3112965162 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f967c0b8470 msgr2=0x7f967c14e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.434+0000 7f967b400640 1 --2- 172.21.15.26:0/3112965162 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f967c0b8470 0x7f967c14e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.434+0000 7f967b400640 1 -- 172.21.15.26:0/3112965162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f967c0d1980 msgr2=0x7f967c0c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.434+0000 7f967b400640 1 --2- 172.21.15.26:0/3112965162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f967c0d1980 0x7f967c0c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.434+0000 7f967b400640 1 -- 172.21.15.26:0/3112965162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f967c0b1f80 con 0x7f967c0d1d60 2024-04-26T21:08:30.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.434+0000 7f967aa00640 1 --2- 172.21.15.26:0/3112965162 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f967c0b8470 0x7f967c14e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:08:30.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.434+0000 7f967a000640 1 --2- 172.21.15.26:0/3112965162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f967c0d1980 0x7f967c0c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:30.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.434+0000 7f967b400640 1 --2- 172.21.15.26:0/3112965162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f967c0d1d60 0x7f967c0c9fb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9674009dd0 tx=0x7f967400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:30.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.435+0000 7f9672a00640 1 -- 172.21.15.26:0/3112965162 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9674018020 con 0x7f967c0d1d60 2024-04-26T21:08:30.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.435+0000 7f9672a00640 1 -- 172.21.15.26:0/3112965162 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9674014480 con 0x7f967c0d1d60 2024-04-26T21:08:30.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.435+0000 7f967be00640 1 -- 172.21.15.26:0/3112965162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f967c0ca5e0 con 0x7f967c0d1d60 2024-04-26T21:08:30.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.435+0000 7f9672a00640 1 -- 172.21.15.26:0/3112965162 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9674014870 con 0x7f967c0d1d60 2024-04-26T21:08:30.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.435+0000 7f967be00640 1 -- 172.21.15.26:0/3112965162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f967c0caba0 con 0x7f967c0d1d60 2024-04-26T21:08:30.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.436+0000 7f967be00640 1 -- 172.21.15.26:0/3112965162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f967c0506c0 con 0x7f967c0d1d60 2024-04-26T21:08:30.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.437+0000 7f9672a00640 1 -- 172.21.15.26:0/3112965162 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f9674005d10 con 0x7f967c0d1d60 2024-04-26T21:08:30.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.437+0000 7f9672a00640 1 --2- 172.21.15.26:0/3112965162 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f965c06c860 0x7f965c06ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:30.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.438+0000 7f9672a00640 1 -- 172.21.15.26:0/3112965162 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7f967408c1d0 con 0x7f967c0d1d60 2024-04-26T21:08:30.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.438+0000 7f967aa00640 1 --2- 172.21.15.26:0/3112965162 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f965c06c860 0x7f965c06ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:30.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.438+0000 7f967aa00640 1 --2- 172.21.15.26:0/3112965162 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f965c06c860 0x7f965c06ed20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f966c007fd0 tx=0x7f966c007d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:30.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.442+0000 7f9672a00640 1 -- 172.21.15.26:0/3112965162 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f967c0506c0 con 0x7f967c0d1d60 2024-04-26T21:08:30.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.648+0000 7f967be00640 1 -- 172.21.15.26:0/3112965162 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f967c0d2760 con 0x7f967c0d1d60 2024-04-26T21:08:30.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.649+0000 7f9672a00640 1 -- 172.21.15.26:0/3112965162 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10861 (secure 0 0 0) 0x7f9674022070 con 0x7f967c0d1d60 2024-04-26T21:08:30.650 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:30.650 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":13,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:08:30.006391+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:08:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.652+0000 7f967be00640 1 -- 172.21.15.26:0/3112965162 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f965c06c860 msgr2=0x7f965c06ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.652+0000 7f967be00640 1 --2- 172.21.15.26:0/3112965162 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f965c06c860 0x7f965c06ed20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f966c007fd0 tx=0x7f966c007d80 comp rx=0 tx=0).stop 2024-04-26T21:08:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.652+0000 7f967be00640 1 -- 172.21.15.26:0/3112965162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f967c0d1d60 msgr2=0x7f967c0c9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.652+0000 7f967be00640 1 --2- 172.21.15.26:0/3112965162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f967c0d1d60 0x7f967c0c9fb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9674009dd0 tx=0x7f967400c490 comp rx=0 tx=0).stop 2024-04-26T21:08:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.652+0000 7f967be00640 1 -- 172.21.15.26:0/3112965162 shutdown_connections 2024-04-26T21:08:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.652+0000 7f967be00640 1 --2- 172.21.15.26:0/3112965162 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f965c06c860 0x7f965c06ed20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.652+0000 7f967be00640 1 --2- 172.21.15.26:0/3112965162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f967c0d1d60 0x7f967c0c9fb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.652+0000 7f967be00640 1 --2- 172.21.15.26:0/3112965162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f967c0d1980 0x7f967c0c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.652+0000 7f967be00640 1 --2- 172.21.15.26:0/3112965162 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f967c0b8470 0x7f967c14e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.652+0000 7f967be00640 1 -- 172.21.15.26:0/3112965162 >> 172.21.15.26:0/3112965162 conn(0x7f967c09e410 msgr2=0x7f967c0d4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.652+0000 7f967be00640 1 -- 172.21.15.26:0/3112965162 shutdown_connections 2024-04-26T21:08:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.652+0000 7f967be00640 1 -- 172.21.15.26:0/3112965162 wait complete. 2024-04-26T21:08:30.670 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-04-26T21:08:30.670 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:08:30.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.874+0000 7f0426800640 1 Processor -- start 2024-04-26T21:08:30.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.874+0000 7f0426800640 1 -- start start 2024-04-26T21:08:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.876+0000 7f0426800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0420145c20 0x7f04200d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.876+0000 7f0426800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0420146450 0x7f04200d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.876+0000 7f0426800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f04201468b0 0x7f04200d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.876+0000 7f0426800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f04200ced10 con 0x7f04201468b0 2024-04-26T21:08:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.876+0000 7f0426800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f04200cf690 con 0x7f0420146450 2024-04-26T21:08:30.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.876+0000 7f0426800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f04200cf930 con 0x7f0420145c20 2024-04-26T21:08:30.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.876+0000 7f0425e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f04201468b0 0x7f04200d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:30.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.876+0000 7f0425e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f04201468b0 0x7f04200d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42088/0 (socket says 172.21.15.26:42088) 2024-04-26T21:08:30.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.876+0000 7f0425e00640 1 -- 172.21.15.26:0/1339050584 learned_addr learned my addr 172.21.15.26:0/1339050584 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:30.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.877+0000 7f0424a00640 1 --2- 172.21.15.26:0/1339050584 >> v2:172.21.15.86:3300/0 conn(0x7f0420146450 0x7f04200d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:30.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.877+0000 7f0425400640 1 --2- 172.21.15.26:0/1339050584 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0420145c20 0x7f04200d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:30.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.877+0000 7f0425e00640 1 -- 172.21.15.26:0/1339050584 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0420145c20 msgr2=0x7f04200d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:30.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.877+0000 7f0425e00640 1 --2- 172.21.15.26:0/1339050584 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0420145c20 0x7f04200d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.877+0000 7f0425e00640 1 -- 172.21.15.26:0/1339050584 >> v2:172.21.15.86:3300/0 conn(0x7f0420146450 msgr2=0x7f04200d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:30.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.877+0000 7f0425e00640 1 --2- 172.21.15.26:0/1339050584 >> v2:172.21.15.86:3300/0 conn(0x7f0420146450 0x7f04200d1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.877+0000 7f0425e00640 1 -- 172.21.15.26:0/1339050584 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04200cfc10 con 0x7f04201468b0 2024-04-26T21:08:30.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.878+0000 7f0425e00640 1 --2- 172.21.15.26:0/1339050584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04201468b0 0x7f04200d1670 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f04200d1d70 tx=0x7f041802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5290dd63ff88df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:30.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.878+0000 7f041fe00640 1 -- 172.21.15.26:0/1339050584 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418036070 con 0x7f04201468b0 2024-04-26T21:08:30.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.878+0000 7f041fe00640 1 -- 172.21.15.26:0/1339050584 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0418031600 con 0x7f04201468b0 2024-04-26T21:08:30.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.878+0000 7f041fe00640 1 -- 172.21.15.26:0/1339050584 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04180319d0 con 0x7f04201468b0 2024-04-26T21:08:30.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.879+0000 7f0426800640 1 -- 172.21.15.26:0/1339050584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04201468b0 msgr2=0x7f04200d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:30.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.879+0000 7f0426800640 1 --2- 172.21.15.26:0/1339050584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04201468b0 0x7f04200d1670 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f04200d1d70 tx=0x7f041802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.879+0000 7f0426800640 1 -- 172.21.15.26:0/1339050584 shutdown_connections 2024-04-26T21:08:30.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.879+0000 7f0426800640 1 --2- 172.21.15.26:0/1339050584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04201468b0 0x7f04200d1670 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.879+0000 7f0426800640 1 --2- 172.21.15.26:0/1339050584 >> v2:172.21.15.86:3300/0 conn(0x7f0420146450 0x7f04200d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.879+0000 7f0426800640 1 --2- 172.21.15.26:0/1339050584 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0420145c20 0x7f04200d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.879+0000 7f0426800640 1 -- 172.21.15.26:0/1339050584 >> 172.21.15.26:0/1339050584 conn(0x7f042006c690 msgr2=0x7f042006caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:30.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.879+0000 7f0426800640 1 -- 172.21.15.26:0/1339050584 shutdown_connections 2024-04-26T21:08:30.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.879+0000 7f0426800640 1 -- 172.21.15.26:0/1339050584 wait complete. 2024-04-26T21:08:30.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.881+0000 7f0426800640 1 Processor -- start 2024-04-26T21:08:30.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.881+0000 7f0426800640 1 -- start start 2024-04-26T21:08:30.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.882+0000 7f0426800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0420145c20 0x7f0420159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:30.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.882+0000 7f0426800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0420146450 0x7f04200b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:30.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.882+0000 7f0426800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04201468b0 0x7f04201336f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:30.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.882+0000 7f0426800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0420146000 con 0x7f04201468b0 2024-04-26T21:08:30.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.882+0000 7f0426800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04200d01d0 con 0x7f0420146450 2024-04-26T21:08:30.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.882+0000 7f0426800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f042015c7b0 con 0x7f0420145c20 2024-04-26T21:08:30.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.882+0000 7f0425e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04201468b0 0x7f04201336f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:30.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.882+0000 7f0425e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04201468b0 0x7f04201336f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42102/0 (socket says 172.21.15.26:42102) 2024-04-26T21:08:30.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.882+0000 7f0425e00640 1 -- 172.21.15.26:0/2487472210 learned_addr learned my addr 172.21.15.26:0/2487472210 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:30.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.882+0000 7f0425e00640 1 -- 172.21.15.26:0/2487472210 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0420145c20 msgr2=0x7f0420159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:30.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.882+0000 7f0425400640 1 --2- 172.21.15.26:0/2487472210 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0420145c20 0x7f0420159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:30.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.882+0000 7f0425e00640 1 --2- 172.21.15.26:0/2487472210 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0420145c20 0x7f0420159450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.882+0000 7f0425e00640 1 -- 172.21.15.26:0/2487472210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0420146450 msgr2=0x7f04200b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:30.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.882+0000 7f0425e00640 1 --2- 172.21.15.26:0/2487472210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0420146450 0x7f04200b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:30.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.882+0000 7f0425e00640 1 -- 172.21.15.26:0/2487472210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042015c1f0 con 0x7f04201468b0 2024-04-26T21:08:30.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.883+0000 7f0425e00640 1 --2- 172.21.15.26:0/2487472210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04201468b0 0x7f04201336f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f041802ac90 tx=0x7f0418004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:30.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.883+0000 7f041d600640 1 -- 172.21.15.26:0/2487472210 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041803f020 con 0x7f04201468b0 2024-04-26T21:08:30.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.883+0000 7f041d600640 1 -- 172.21.15.26:0/2487472210 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0418037040 con 0x7f04201468b0 2024-04-26T21:08:30.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.883+0000 7f041d600640 1 -- 172.21.15.26:0/2487472210 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418036070 con 0x7f04201468b0 2024-04-26T21:08:30.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.883+0000 7f0426800640 1 -- 172.21.15.26:0/2487472210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042015c4d0 con 0x7f04201468b0 2024-04-26T21:08:30.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.883+0000 7f0426800640 1 -- 172.21.15.26:0/2487472210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04200b42f0 con 0x7f04201468b0 2024-04-26T21:08:30.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.884+0000 7f0426800640 1 -- 172.21.15.26:0/2487472210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f042006a700 con 0x7f04201468b0 2024-04-26T21:08:30.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.885+0000 7f041d600640 1 -- 172.21.15.26:0/2487472210 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f0418042020 con 0x7f04201468b0 2024-04-26T21:08:30.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.886+0000 7f041d600640 1 --2- 172.21.15.26:0/2487472210 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f040006c860 0x7f040006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:30.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.886+0000 7f041d600640 1 -- 172.21.15.26:0/2487472210 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7f04180abdc0 con 0x7f04201468b0 2024-04-26T21:08:30.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.886+0000 7f0425400640 1 --2- 172.21.15.26:0/2487472210 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f040006c860 0x7f040006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:30.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.887+0000 7f0425400640 1 --2- 172.21.15.26:0/2487472210 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f040006c860 0x7f040006ed20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f041400b7e0 tx=0x7f0414008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:30.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:30.890+0000 7f041d600640 1 -- 172.21.15.26:0/2487472210 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0418049020 con 0x7f04201468b0 2024-04-26T21:08:31.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:31.137+0000 7f0426800640 1 -- 172.21.15.26:0/2487472210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0420070150 con 0x7f04201468b0 2024-04-26T21:08:31.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:31.138+0000 7f041d600640 1 -- 172.21.15.26:0/2487472210 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f041807a7a0 con 0x7f04201468b0 2024-04-26T21:08:31.139 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:31.139 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/2695371801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2695371801},{"type":"v1","addr":"172.21.15.26:6836","nonce":2695371801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T21:08:31.139 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 2 2024-04-26T21:08:31.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:31.141+0000 7f0426800640 1 -- 172.21.15.26:0/2487472210 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f040006c860 msgr2=0x7f040006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:31.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:31.141+0000 7f0426800640 1 --2- 172.21.15.26:0/2487472210 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f040006c860 0x7f040006ed20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f041400b7e0 tx=0x7f0414008040 comp rx=0 tx=0).stop 2024-04-26T21:08:31.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:31.141+0000 7f0426800640 1 -- 172.21.15.26:0/2487472210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04201468b0 msgr2=0x7f04201336f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:31.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:31.141+0000 7f0426800640 1 --2- 172.21.15.26:0/2487472210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04201468b0 0x7f04201336f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f041802ac90 tx=0x7f0418004a40 comp rx=0 tx=0).stop 2024-04-26T21:08:31.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:31.141+0000 7f0426800640 1 -- 172.21.15.26:0/2487472210 shutdown_connections 2024-04-26T21:08:31.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:31.141+0000 7f0426800640 1 --2- 172.21.15.26:0/2487472210 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f040006c860 0x7f040006ed20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:31.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:31.141+0000 7f0426800640 1 --2- 172.21.15.26:0/2487472210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04201468b0 0x7f04201336f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:31.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:31.141+0000 7f0426800640 1 --2- 172.21.15.26:0/2487472210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0420146450 0x7f04200b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:31.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:31.141+0000 7f0426800640 1 --2- 172.21.15.26:0/2487472210 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0420145c20 0x7f0420159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:31.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:31.141+0000 7f0426800640 1 -- 172.21.15.26:0/2487472210 >> 172.21.15.26:0/2487472210 conn(0x7f042006c690 msgr2=0x7f0420073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:31.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:31.141+0000 7f0426800640 1 -- 172.21.15.26:0/2487472210 shutdown_connections 2024-04-26T21:08:31.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:31.141+0000 7f0426800640 1 -- 172.21.15.26:0/2487472210 wait complete. 2024-04-26T21:08:33.161 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:08:33.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.381+0000 7f82b1400640 1 Processor -- start 2024-04-26T21:08:33.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.382+0000 7f82b1400640 1 -- start start 2024-04-26T21:08:33.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.383+0000 7f82b1400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f82ac145db0 0x7f82ac0728a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:33.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.383+0000 7f82b1400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82ac0d3aa0 0x7f82ac0b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:33.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.383+0000 7f82b1400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82ac15da10 con 0x7f82ac15d580 2024-04-26T21:08:33.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.383+0000 7f82b1400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f82ac0b6920 con 0x7f82ac145db0 2024-04-26T21:08:33.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.383+0000 7f82b1400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f82ac0d1830 con 0x7f82ac0d3aa0 2024-04-26T21:08:33.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.383+0000 7f82abe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f82ac145db0 0x7f82ac0728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:33.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.384+0000 7f82abe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f82ac145db0 0x7f82ac0728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42108/0 (socket says 172.21.15.26:42108) 2024-04-26T21:08:33.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.384+0000 7f82abe00640 1 -- 172.21.15.26:0/1501095363 learned_addr learned my addr 172.21.15.26:0/1501095363 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:33.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.384+0000 7f82abe00640 1 -- 172.21.15.26:0/1501095363 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82ac0d3aa0 msgr2=0x7f82ac0b63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:33.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.384+0000 7f82b0a00640 1 --2- 172.21.15.26:0/1501095363 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82ac0d3aa0 0x7f82ac0b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:33.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.384+0000 7f82abe00640 1 --2- 172.21.15.26:0/1501095363 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82ac0d3aa0 0x7f82ac0b63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.384+0000 7f82abe00640 1 -- 172.21.15.26:0/1501095363 >> v1:172.21.15.86:6789/0 conn(0x7f82ac15d580 legacy=0x7f82ac0d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:33.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.384+0000 7f82abe00640 1 -- 172.21.15.26:0/1501095363 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ac0b6b50 con 0x7f82ac145db0 2024-04-26T21:08:33.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.384+0000 7f82b0a00640 1 --2- 172.21.15.26:0/1501095363 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82ac0d3aa0 0x7f82ac0b63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:33.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.385+0000 7f82abe00640 1 --2- 172.21.15.26:0/1501095363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac145db0 0x7f82ac0728a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f829c009870 tx=0x7f829c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff442203271d777d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:33.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.386+0000 7f82ab400640 1 -- 172.21.15.26:0/1501095363 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c037070 con 0x7f82ac145db0 2024-04-26T21:08:33.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.386+0000 7f82ab400640 1 -- 172.21.15.26:0/1501095363 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f829c032440 con 0x7f82ac145db0 2024-04-26T21:08:33.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.386+0000 7f82ab400640 1 -- 172.21.15.26:0/1501095363 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c032810 con 0x7f82ac145db0 2024-04-26T21:08:33.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.386+0000 7f82b1400640 1 -- 172.21.15.26:0/1501095363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac145db0 msgr2=0x7f82ac0728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:33.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.386+0000 7f82b1400640 1 --2- 172.21.15.26:0/1501095363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac145db0 0x7f82ac0728a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f829c009870 tx=0x7f829c02a210 comp rx=0 tx=0).stop 2024-04-26T21:08:33.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.386+0000 7f82b1400640 1 -- 172.21.15.26:0/1501095363 shutdown_connections 2024-04-26T21:08:33.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.386+0000 7f82b1400640 1 --2- 172.21.15.26:0/1501095363 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82ac0d3aa0 0x7f82ac0b63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.386+0000 7f82b1400640 1 --2- 172.21.15.26:0/1501095363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac145db0 0x7f82ac0728a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.386+0000 7f82b1400640 1 -- 172.21.15.26:0/1501095363 >> 172.21.15.26:0/1501095363 conn(0x7f82ac06c660 msgr2=0x7f82ac06ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:33.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.386+0000 7f82b1400640 1 -- 172.21.15.26:0/1501095363 shutdown_connections 2024-04-26T21:08:33.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.386+0000 7f82b1400640 1 -- 172.21.15.26:0/1501095363 wait complete. 2024-04-26T21:08:33.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.388+0000 7f82b1400640 1 Processor -- start 2024-04-26T21:08:33.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.388+0000 7f82b1400640 1 -- start start 2024-04-26T21:08:33.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.389+0000 7f82b1400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82ac0d3aa0 0x7f82ac15c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:33.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.389+0000 7f82b1400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac145db0 0x7f82ac1f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:33.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.389+0000 7f82b1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ac15d580 0x7f82ac0b3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:33.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.389+0000 7f82b1400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac0b73f0 con 0x7f82ac145db0 2024-04-26T21:08:33.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.389+0000 7f82b1400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac0b7110 con 0x7f82ac15d580 2024-04-26T21:08:33.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.389+0000 7f82b1400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f82ac146190 con 0x7f82ac0d3aa0 2024-04-26T21:08:33.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.390+0000 7f82a3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac145db0 0x7f82ac1f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:33.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.390+0000 7f82a3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac145db0 0x7f82ac1f4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42122/0 (socket says 172.21.15.26:42122) 2024-04-26T21:08:33.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.390+0000 7f82a3400640 1 -- 172.21.15.26:0/2444414916 learned_addr learned my addr 172.21.15.26:0/2444414916 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:33.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.390+0000 7f82b0a00640 1 --2- 172.21.15.26:0/2444414916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ac15d580 0x7f82ac0b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:33.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.390+0000 7f82abe00640 1 --2- 172.21.15.26:0/2444414916 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82ac0d3aa0 0x7f82ac15c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:33.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.390+0000 7f82a3400640 1 -- 172.21.15.26:0/2444414916 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82ac0d3aa0 msgr2=0x7f82ac15c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:33.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.390+0000 7f82a3400640 1 --2- 172.21.15.26:0/2444414916 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82ac0d3aa0 0x7f82ac15c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.390+0000 7f82a3400640 1 -- 172.21.15.26:0/2444414916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ac15d580 msgr2=0x7f82ac0b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:33.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.390+0000 7f82a3400640 1 --2- 172.21.15.26:0/2444414916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ac15d580 0x7f82ac0b3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.390+0000 7f82a3400640 1 -- 172.21.15.26:0/2444414916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ac1f5920 con 0x7f82ac145db0 2024-04-26T21:08:33.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.390+0000 7f82b0a00640 1 --2- 172.21.15.26:0/2444414916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ac15d580 0x7f82ac0b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:33.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.390+0000 7f82a3400640 1 --2- 172.21.15.26:0/2444414916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac145db0 0x7f82ac1f4f80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f829400b3f0 tx=0x7f829400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:33.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.391+0000 7f82a8c00640 1 -- 172.21.15.26:0/2444414916 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294015020 con 0x7f82ac145db0 2024-04-26T21:08:33.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.391+0000 7f82a8c00640 1 -- 172.21.15.26:0/2444414916 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8294011830 con 0x7f82ac145db0 2024-04-26T21:08:33.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.391+0000 7f82b1400640 1 -- 172.21.15.26:0/2444414916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ac1f5c00 con 0x7f82ac145db0 2024-04-26T21:08:33.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.391+0000 7f82a8c00640 1 -- 172.21.15.26:0/2444414916 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294011c20 con 0x7f82ac145db0 2024-04-26T21:08:33.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.391+0000 7f82b1400640 1 -- 172.21.15.26:0/2444414916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ac0cbbb0 con 0x7f82ac145db0 2024-04-26T21:08:33.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.392+0000 7f82b1400640 1 -- 172.21.15.26:0/2444414916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82ac0cc0a0 con 0x7f82ac145db0 2024-04-26T21:08:33.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.393+0000 7f82a8c00640 1 -- 172.21.15.26:0/2444414916 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f8294018020 con 0x7f82ac145db0 2024-04-26T21:08:33.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.394+0000 7f82a8c00640 1 --2- 172.21.15.26:0/2444414916 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f828806c860 0x7f828806ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:33.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.394+0000 7f82a8c00640 1 -- 172.21.15.26:0/2444414916 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f829408b470 con 0x7f82ac145db0 2024-04-26T21:08:33.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.394+0000 7f82abe00640 1 --2- 172.21.15.26:0/2444414916 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f828806c860 0x7f828806ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:33.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.395+0000 7f82abe00640 1 --2- 172.21.15.26:0/2444414916 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f828806c860 0x7f828806ed20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f829c034000 tx=0x7f829c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.400+0000 7f82a8c00640 1 -- 172.21.15.26:0/2444414916 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f829401f030 con 0x7f82ac145db0 2024-04-26T21:08:33.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.665+0000 7f82b1400640 1 -- 172.21.15.26:0/2444414916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82ac0b26d0 con 0x7f82ac145db0 2024-04-26T21:08:33.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.665+0000 7f82a8c00640 1 -- 172.21.15.26:0/2444414916 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3864 (secure 0 0 0) 0x7f82ac0b26d0 con 0x7f82ac145db0 2024-04-26T21:08:33.666 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:33.666 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/2695371801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2695371801},{"type":"v1","addr":"172.21.15.26:6836","nonce":2695371801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3349732696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3349732696},{"type":"v1","addr":"172.21.15.26:6837","nonce":3349732696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/2288633870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2288633870},{"type":"v1","addr":"172.21.15.86:6833","nonce":2288633870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/2125360615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2125360615},{"type":"v1","addr":"172.21.15.86:6835","nonce":2125360615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:08:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 3 2024-04-26T21:08:33.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.668+0000 7f82b1400640 1 -- 172.21.15.26:0/2444414916 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f828806c860 msgr2=0x7f828806ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:33.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.668+0000 7f82b1400640 1 --2- 172.21.15.26:0/2444414916 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f828806c860 0x7f828806ed20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f829c034000 tx=0x7f829c02e000 comp rx=0 tx=0).stop 2024-04-26T21:08:33.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.668+0000 7f82b1400640 1 -- 172.21.15.26:0/2444414916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac145db0 msgr2=0x7f82ac1f4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:33.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.668+0000 7f82b1400640 1 --2- 172.21.15.26:0/2444414916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac145db0 0x7f82ac1f4f80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f829400b3f0 tx=0x7f829400b8b0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.669+0000 7f82b1400640 1 -- 172.21.15.26:0/2444414916 shutdown_connections 2024-04-26T21:08:33.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.669+0000 7f82b1400640 1 --2- 172.21.15.26:0/2444414916 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f828806c860 0x7f828806ed20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.669+0000 7f82b1400640 1 --2- 172.21.15.26:0/2444414916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82ac15d580 0x7f82ac0b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.669+0000 7f82b1400640 1 --2- 172.21.15.26:0/2444414916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac145db0 0x7f82ac1f4f80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.669+0000 7f82b1400640 1 --2- 172.21.15.26:0/2444414916 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82ac0d3aa0 0x7f82ac15c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.669+0000 7f82b1400640 1 -- 172.21.15.26:0/2444414916 >> 172.21.15.26:0/2444414916 conn(0x7f82ac06c660 msgr2=0x7f82ac0b0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:33.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.669+0000 7f82b1400640 1 -- 172.21.15.26:0/2444414916 shutdown_connections 2024-04-26T21:08:33.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.669+0000 7f82b1400640 1 -- 172.21.15.26:0/2444414916 wait complete. 2024-04-26T21:08:33.690 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:08:33.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.891+0000 7f410ac00640 1 Processor -- start 2024-04-26T21:08:33.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.891+0000 7f410ac00640 1 -- start start 2024-04-26T21:08:33.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.893+0000 7f410ac00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f41040d39f0 0x7f41040d5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:33.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.893+0000 7f410ac00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f41040d3dd0 0x7f41040d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:33.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.893+0000 7f410ac00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f410415f000 0x7f410415f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:33.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.893+0000 7f410ac00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f41040d0ef0 con 0x7f41040d3dd0 2024-04-26T21:08:33.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.893+0000 7f410ac00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f41040b4900 con 0x7f410415f000 2024-04-26T21:08:33.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.893+0000 7f410ac00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f41040b4be0 con 0x7f41040d39f0 2024-04-26T21:08:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.893+0000 7f4103400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f41040d3dd0 0x7f41040d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.893+0000 7f4103400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f41040d3dd0 0x7f41040d07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42156/0 (socket says 172.21.15.26:42156) 2024-04-26T21:08:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.893+0000 7f4103400640 1 -- 172.21.15.26:0/4153953508 learned_addr learned my addr 172.21.15.26:0/4153953508 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.894+0000 7f4108c00640 1 --2- 172.21.15.26:0/4153953508 >> v2:172.21.15.86:3300/0 conn(0x7f410415f000 0x7f410415f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.894+0000 7f4103e00640 1 --2- 172.21.15.26:0/4153953508 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f41040d39f0 0x7f41040d5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.894+0000 7f4103400640 1 -- 172.21.15.26:0/4153953508 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f41040d39f0 msgr2=0x7f41040d5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.894+0000 7f4103400640 1 --2- 172.21.15.26:0/4153953508 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f41040d39f0 0x7f41040d5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.894+0000 7f4103400640 1 -- 172.21.15.26:0/4153953508 >> v2:172.21.15.86:3300/0 conn(0x7f410415f000 msgr2=0x7f410415f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.894+0000 7f4103400640 1 --2- 172.21.15.26:0/4153953508 >> v2:172.21.15.86:3300/0 conn(0x7f410415f000 0x7f410415f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.894+0000 7f4103400640 1 -- 172.21.15.26:0/4153953508 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41040b4ec0 con 0x7f41040d3dd0 2024-04-26T21:08:33.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.895+0000 7f4103400640 1 --2- 172.21.15.26:0/4153953508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41040d3dd0 0x7f41040d07c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f41040d2c30 tx=0x7f40f002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799ab354d07e1a8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:33.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.895+0000 7f4102a00640 1 -- 172.21.15.26:0/4153953508 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0037070 con 0x7f41040d3dd0 2024-04-26T21:08:33.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.895+0000 7f4102a00640 1 -- 172.21.15.26:0/4153953508 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f0032440 con 0x7f41040d3dd0 2024-04-26T21:08:33.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.895+0000 7f4102a00640 1 -- 172.21.15.26:0/4153953508 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0032810 con 0x7f41040d3dd0 2024-04-26T21:08:33.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.896+0000 7f410ac00640 1 -- 172.21.15.26:0/4153953508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41040d3dd0 msgr2=0x7f41040d07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:33.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.896+0000 7f410ac00640 1 --2- 172.21.15.26:0/4153953508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41040d3dd0 0x7f41040d07c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f41040d2c30 tx=0x7f40f002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.896+0000 7f410ac00640 1 -- 172.21.15.26:0/4153953508 shutdown_connections 2024-04-26T21:08:33.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.896+0000 7f410ac00640 1 --2- 172.21.15.26:0/4153953508 >> v2:172.21.15.86:3300/0 conn(0x7f410415f000 0x7f410415f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.896+0000 7f410ac00640 1 --2- 172.21.15.26:0/4153953508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41040d3dd0 0x7f41040d07c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.896+0000 7f410ac00640 1 --2- 172.21.15.26:0/4153953508 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f41040d39f0 0x7f41040d5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.896+0000 7f410ac00640 1 -- 172.21.15.26:0/4153953508 >> 172.21.15.26:0/4153953508 conn(0x7f41040d51a0 msgr2=0x7f41040d55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:33.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.896+0000 7f410ac00640 1 -- 172.21.15.26:0/4153953508 shutdown_connections 2024-04-26T21:08:33.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.896+0000 7f410ac00640 1 -- 172.21.15.26:0/4153953508 wait complete. 2024-04-26T21:08:33.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.898+0000 7f410ac00640 1 Processor -- start 2024-04-26T21:08:33.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.899+0000 7f410ac00640 1 -- start start 2024-04-26T21:08:33.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.900+0000 7f410ac00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f41040d39f0 0x7f410415da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:33.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.900+0000 7f410ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41040d3dd0 0x7f41040ce760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:33.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.900+0000 7f410ac00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f410415f000 0x7f41040ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:33.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.900+0000 7f410ac00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f41040d1950 con 0x7f410415f000 2024-04-26T21:08:33.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.900+0000 7f410ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41040d17d0 con 0x7f41040d3dd0 2024-04-26T21:08:33.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.900+0000 7f410ac00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f41040d1ad0 con 0x7f41040d39f0 2024-04-26T21:08:33.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.900+0000 7f4108c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f410415f000 0x7f41040ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:33.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.900+0000 7f4108c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f410415f000 0x7f41040ceca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42170/0 (socket says 172.21.15.26:42170) 2024-04-26T21:08:33.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.900+0000 7f4108c00640 1 -- 172.21.15.26:0/1135401600 learned_addr learned my addr 172.21.15.26:0/1135401600 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:33.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.900+0000 7f4103400640 1 --2- 172.21.15.26:0/1135401600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41040d3dd0 0x7f41040ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:33.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.900+0000 7f4108c00640 1 -- 172.21.15.26:0/1135401600 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f41040d39f0 msgr2=0x7f410415da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:33.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.900+0000 7f4103e00640 1 --2- 172.21.15.26:0/1135401600 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f41040d39f0 0x7f410415da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:33.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.900+0000 7f4108c00640 1 --2- 172.21.15.26:0/1135401600 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f41040d39f0 0x7f410415da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.900+0000 7f4108c00640 1 -- 172.21.15.26:0/1135401600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41040d3dd0 msgr2=0x7f41040ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:33.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.900+0000 7f4108c00640 1 --2- 172.21.15.26:0/1135401600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41040d3dd0 0x7f41040ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:33.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.901+0000 7f4108c00640 1 -- 172.21.15.26:0/1135401600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f410415df90 con 0x7f410415f000 2024-04-26T21:08:33.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.901+0000 7f4103400640 1 --2- 172.21.15.26:0/1135401600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41040d3dd0 0x7f41040ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:33.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.901+0000 7f4103e00640 1 --2- 172.21.15.26:0/1135401600 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f41040d39f0 0x7f410415da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:33.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.901+0000 7f4108c00640 1 --2- 172.21.15.26:0/1135401600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f410415f000 0x7f41040ceca0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f40fc00d5d0 tx=0x7f40fc00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:33.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.901+0000 7f40fbe00640 1 -- 172.21.15.26:0/1135401600 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc0125f0 con 0x7f410415f000 2024-04-26T21:08:33.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.901+0000 7f40fbe00640 1 -- 172.21.15.26:0/1135401600 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40fc019070 con 0x7f410415f000 2024-04-26T21:08:33.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.901+0000 7f40fbe00640 1 -- 172.21.15.26:0/1135401600 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc0129f0 con 0x7f410415f000 2024-04-26T21:08:33.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.901+0000 7f410ac00640 1 -- 172.21.15.26:0/1135401600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41041605d0 con 0x7f410415f000 2024-04-26T21:08:33.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.901+0000 7f410ac00640 1 -- 172.21.15.26:0/1135401600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4104160b90 con 0x7f410415f000 2024-04-26T21:08:33.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.903+0000 7f410ac00640 1 -- 172.21.15.26:0/1135401600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f410406a700 con 0x7f410415f000 2024-04-26T21:08:33.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.904+0000 7f40fbe00640 1 -- 172.21.15.26:0/1135401600 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f40fc012b90 con 0x7f410415f000 2024-04-26T21:08:33.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.905+0000 7f40fbe00640 1 --2- 172.21.15.26:0/1135401600 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f40e406c860 0x7f40e406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:33.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.905+0000 7f40fbe00640 1 -- 172.21.15.26:0/1135401600 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f40fc08bb60 con 0x7f410415f000 2024-04-26T21:08:33.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.906+0000 7f4103e00640 1 --2- 172.21.15.26:0/1135401600 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f40e406c860 0x7f40e406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:33.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.906+0000 7f4103e00640 1 --2- 172.21.15.26:0/1135401600 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f40e406c860 0x7f40e406ed20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009f70 tx=0x7f40f400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:33.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:33.911+0000 7f40fbe00640 1 -- 172.21.15.26:0/1135401600 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f40fc05a330 con 0x7f410415f000 2024-04-26T21:08:34.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.125+0000 7f410ac00640 1 -- 172.21.15.26:0/1135401600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41040d4f80 con 0x7f410415f000 2024-04-26T21:08:34.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.127+0000 7f40fbe00640 1 -- 172.21.15.26:0/1135401600 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12836 (secure 0 0 0) 0x7f40fc059c30 con 0x7f410415f000 2024-04-26T21:08:34.128 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:34.128 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:08:34.036538+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:08:34.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.130+0000 7f410ac00640 1 -- 172.21.15.26:0/1135401600 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f40e406c860 msgr2=0x7f40e406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:34.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.130+0000 7f410ac00640 1 --2- 172.21.15.26:0/1135401600 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f40e406c860 0x7f40e406ed20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009f70 tx=0x7f40f400b040 comp rx=0 tx=0).stop 2024-04-26T21:08:34.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.130+0000 7f410ac00640 1 -- 172.21.15.26:0/1135401600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f410415f000 msgr2=0x7f41040ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:34.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.130+0000 7f410ac00640 1 --2- 172.21.15.26:0/1135401600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f410415f000 0x7f41040ceca0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f40fc00d5d0 tx=0x7f40fc00da90 comp rx=0 tx=0).stop 2024-04-26T21:08:34.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.130+0000 7f410ac00640 1 -- 172.21.15.26:0/1135401600 shutdown_connections 2024-04-26T21:08:34.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.130+0000 7f410ac00640 1 --2- 172.21.15.26:0/1135401600 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f40e406c860 0x7f40e406ed20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.130+0000 7f410ac00640 1 --2- 172.21.15.26:0/1135401600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f410415f000 0x7f41040ceca0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.130+0000 7f410ac00640 1 --2- 172.21.15.26:0/1135401600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41040d3dd0 0x7f41040ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.130+0000 7f410ac00640 1 --2- 172.21.15.26:0/1135401600 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f41040d39f0 0x7f410415da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.130+0000 7f410ac00640 1 -- 172.21.15.26:0/1135401600 >> 172.21.15.26:0/1135401600 conn(0x7f41040d51a0 msgr2=0x7f410406e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:34.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.130+0000 7f410ac00640 1 -- 172.21.15.26:0/1135401600 shutdown_connections 2024-04-26T21:08:34.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.130+0000 7f410ac00640 1 -- 172.21.15.26:0/1135401600 wait complete. 2024-04-26T21:08:34.149 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:08:31.322511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:08:34.149 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:08:34.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.333+0000 7fa88a000640 1 Processor -- start 2024-04-26T21:08:34.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.333+0000 7fa88a000640 1 -- start start 2024-04-26T21:08:34.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.335+0000 7fa88a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa8840b5d30 0x7fa8840b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:34.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.335+0000 7fa88a000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa8840b42c0 0x7fa8840b6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:34.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.335+0000 7fa88a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8840b6650 0x7fa88406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:34.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.335+0000 7fa88a000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fa884073700 con 0x7fa8840b42c0 2024-04-26T21:08:34.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.335+0000 7fa88a000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa88406e420 con 0x7fa8840b5d30 2024-04-26T21:08:34.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.335+0000 7fa88a000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa884146c00 con 0x7fa8840b6650 2024-04-26T21:08:34.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.335+0000 7fa87be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa8840b42c0 0x7fa8840b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:34.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.335+0000 7fa87be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa8840b42c0 0x7fa8840b6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42176/0 (socket says 172.21.15.26:42176) 2024-04-26T21:08:34.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.335+0000 7fa87be00640 1 -- 172.21.15.26:0/2096169473 learned_addr learned my addr 172.21.15.26:0/2096169473 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:34.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.335+0000 7fa883e00640 1 --2- 172.21.15.26:0/2096169473 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8840b6650 0x7fa88406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:34.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.335+0000 7fa883400640 1 --2- 172.21.15.26:0/2096169473 >> v2:172.21.15.86:3300/0 conn(0x7fa8840b5d30 0x7fa8840b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:34.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.336+0000 7fa87be00640 1 -- 172.21.15.26:0/2096169473 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8840b6650 msgr2=0x7fa88406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:34.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.336+0000 7fa87be00640 1 --2- 172.21.15.26:0/2096169473 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8840b6650 0x7fa88406e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.336+0000 7fa87be00640 1 -- 172.21.15.26:0/2096169473 >> v2:172.21.15.86:3300/0 conn(0x7fa8840b5d30 msgr2=0x7fa8840b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:34.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.336+0000 7fa87be00640 1 --2- 172.21.15.26:0/2096169473 >> v2:172.21.15.86:3300/0 conn(0x7fa8840b5d30 0x7fa8840b3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.336+0000 7fa87be00640 1 -- 172.21.15.26:0/2096169473 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa884072e60 con 0x7fa8840b42c0 2024-04-26T21:08:34.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.337+0000 7fa87be00640 1 --2- 172.21.15.26:0/2096169473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8840b42c0 0x7fa8840b6110 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa8840700b0 tx=0x7fa86c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65732e1f43e6e244 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:34.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.337+0000 7fa882a00640 1 -- 172.21.15.26:0/2096169473 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c037070 con 0x7fa8840b42c0 2024-04-26T21:08:34.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.337+0000 7fa882a00640 1 -- 172.21.15.26:0/2096169473 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa86c032440 con 0x7fa8840b42c0 2024-04-26T21:08:34.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.337+0000 7fa882a00640 1 -- 172.21.15.26:0/2096169473 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c032810 con 0x7fa8840b42c0 2024-04-26T21:08:34.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.338+0000 7fa88a000640 1 -- 172.21.15.26:0/2096169473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8840b42c0 msgr2=0x7fa8840b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:34.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.338+0000 7fa88a000640 1 --2- 172.21.15.26:0/2096169473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8840b42c0 0x7fa8840b6110 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa8840700b0 tx=0x7fa86c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.338+0000 7fa88a000640 1 -- 172.21.15.26:0/2096169473 shutdown_connections 2024-04-26T21:08:34.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.338+0000 7fa88a000640 1 --2- 172.21.15.26:0/2096169473 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8840b6650 0x7fa88406e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.338+0000 7fa88a000640 1 --2- 172.21.15.26:0/2096169473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8840b42c0 0x7fa8840b6110 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.338+0000 7fa88a000640 1 --2- 172.21.15.26:0/2096169473 >> v2:172.21.15.86:3300/0 conn(0x7fa8840b5d30 0x7fa8840b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.338+0000 7fa88a000640 1 -- 172.21.15.26:0/2096169473 >> 172.21.15.26:0/2096169473 conn(0x7fa8840b2a80 msgr2=0x7fa8840b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:34.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.338+0000 7fa88a000640 1 -- 172.21.15.26:0/2096169473 shutdown_connections 2024-04-26T21:08:34.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.338+0000 7fa88a000640 1 -- 172.21.15.26:0/2096169473 wait complete. 2024-04-26T21:08:34.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.339+0000 7fa88a000640 1 Processor -- start 2024-04-26T21:08:34.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.340+0000 7fa88a000640 1 -- start start 2024-04-26T21:08:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.341+0000 7fa88a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8840b42c0 0x7fa884157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.341+0000 7fa88a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8840b5d30 0x7fa8841f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.341+0000 7fa88a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8840b6650 0x7fa8841ef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.341+0000 7fa88a000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa884072a80 con 0x7fa8840b42c0 2024-04-26T21:08:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.341+0000 7fa88a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa884073420 con 0x7fa8840b6650 2024-04-26T21:08:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.341+0000 7fa88a000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa884072c00 con 0x7fa8840b5d30 2024-04-26T21:08:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.341+0000 7fa883400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8840b42c0 0x7fa884157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.341+0000 7fa883400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8840b42c0 0x7fa884157b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42186/0 (socket says 172.21.15.26:42186) 2024-04-26T21:08:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.341+0000 7fa883400640 1 -- 172.21.15.26:0/2322136820 learned_addr learned my addr 172.21.15.26:0/2322136820 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.341+0000 7fa883400640 1 -- 172.21.15.26:0/2322136820 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8840b5d30 msgr2=0x7fa8841f5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.341+0000 7fa883400640 1 --2- 172.21.15.26:0/2322136820 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8840b5d30 0x7fa8841f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.341+0000 7fa883400640 1 -- 172.21.15.26:0/2322136820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8840b6650 msgr2=0x7fa8841ef0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.341+0000 7fa883400640 1 --2- 172.21.15.26:0/2322136820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8840b6650 0x7fa8841ef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.341+0000 7fa883400640 1 -- 172.21.15.26:0/2322136820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88415b5e0 con 0x7fa8840b42c0 2024-04-26T21:08:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.342+0000 7fa883400640 1 --2- 172.21.15.26:0/2322136820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8840b42c0 0x7fa884157b60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa87400d800 tx=0x7fa87400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.342+0000 7fa87b400640 1 -- 172.21.15.26:0/2322136820 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874018020 con 0x7fa8840b42c0 2024-04-26T21:08:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.342+0000 7fa87b400640 1 -- 172.21.15.26:0/2322136820 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8740096a0 con 0x7fa8840b42c0 2024-04-26T21:08:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.342+0000 7fa87b400640 1 -- 172.21.15.26:0/2322136820 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874009a70 con 0x7fa8840b42c0 2024-04-26T21:08:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.342+0000 7fa88a000640 1 -- 172.21.15.26:0/2322136820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8841f61a0 con 0x7fa8840b42c0 2024-04-26T21:08:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.342+0000 7fa88a000640 1 -- 172.21.15.26:0/2322136820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa884158420 con 0x7fa8840b42c0 2024-04-26T21:08:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.343+0000 7fa88a000640 1 -- 172.21.15.26:0/2322136820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa88406a700 con 0x7fa8840b42c0 2024-04-26T21:08:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.344+0000 7fa87b400640 1 -- 172.21.15.26:0/2322136820 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fa87401b020 con 0x7fa8840b42c0 2024-04-26T21:08:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.346+0000 7fa87b400640 1 --2- 172.21.15.26:0/2322136820 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa86006c860 0x7fa86006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.346+0000 7fa87b400640 1 -- 172.21.15.26:0/2322136820 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fa87408ba00 con 0x7fa8840b42c0 2024-04-26T21:08:34.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.346+0000 7fa87be00640 1 --2- 172.21.15.26:0/2322136820 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa86006c860 0x7fa86006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:34.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.347+0000 7fa87be00640 1 --2- 172.21.15.26:0/2322136820 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa86006c860 0x7fa86006ed20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa86c034000 tx=0x7fa86c02c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:34.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.350+0000 7fa87b400640 1 -- 172.21.15.26:0/2322136820 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa874022030 con 0x7fa8840b42c0 2024-04-26T21:08:34.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.558+0000 7fa88a000640 1 -- 172.21.15.26:0/2322136820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa884072860 con 0x7fa8840b42c0 2024-04-26T21:08:34.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.559+0000 7fa87b400640 1 -- 172.21.15.26:0/2322136820 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7fa87405a1c0 con 0x7fa8840b42c0 2024-04-26T21:08:34.559 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-26T21:08:34.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.561+0000 7fa88a000640 1 -- 172.21.15.26:0/2322136820 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa86006c860 msgr2=0x7fa86006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:34.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.561+0000 7fa88a000640 1 --2- 172.21.15.26:0/2322136820 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa86006c860 0x7fa86006ed20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa86c034000 tx=0x7fa86c02c000 comp rx=0 tx=0).stop 2024-04-26T21:08:34.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.561+0000 7fa88a000640 1 -- 172.21.15.26:0/2322136820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8840b42c0 msgr2=0x7fa884157b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:34.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.561+0000 7fa88a000640 1 --2- 172.21.15.26:0/2322136820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8840b42c0 0x7fa884157b60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa87400d800 tx=0x7fa87400dcc0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.562+0000 7fa88a000640 1 -- 172.21.15.26:0/2322136820 shutdown_connections 2024-04-26T21:08:34.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.562+0000 7fa88a000640 1 --2- 172.21.15.26:0/2322136820 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa86006c860 0x7fa86006ed20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.562+0000 7fa88a000640 1 --2- 172.21.15.26:0/2322136820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8840b6650 0x7fa8841ef0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.562+0000 7fa88a000640 1 --2- 172.21.15.26:0/2322136820 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8840b5d30 0x7fa8841f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.562+0000 7fa88a000640 1 --2- 172.21.15.26:0/2322136820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8840b42c0 0x7fa884157b60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.562+0000 7fa88a000640 1 -- 172.21.15.26:0/2322136820 >> 172.21.15.26:0/2322136820 conn(0x7fa8840b2a80 msgr2=0x7fa88406f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:34.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.562+0000 7fa88a000640 1 -- 172.21.15.26:0/2322136820 shutdown_connections 2024-04-26T21:08:34.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.562+0000 7fa88a000640 1 -- 172.21.15.26:0/2322136820 wait complete. 2024-04-26T21:08:34.581 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:08:34.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.789+0000 7f663b200640 1 Processor -- start 2024-04-26T21:08:34.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.790+0000 7f663b200640 1 -- start start 2024-04-26T21:08:34.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.791+0000 7f663b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f663c0b8f30 0x7f663c0b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:34.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.791+0000 7f663b200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f663c0d0130 con 0x7f663c15f1c0 2024-04-26T21:08:34.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.791+0000 7f663b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f663c0d0410 con 0x7f663c15f5a0 2024-04-26T21:08:34.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.791+0000 7f663b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f663c0d06f0 con 0x7f663c0b8f30 2024-04-26T21:08:34.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.791+0000 7f6639e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f663c15f1c0 0x7f663c0d3b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35438/0 (socket says 172.21.15.26:35438) 2024-04-26T21:08:34.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.792+0000 7f6639e00640 1 -- 172.21.15.26:0/2040870113 learned_addr learned my addr 172.21.15.26:0/2040870113 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:34.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.792+0000 7f663a800640 1 --2- 172.21.15.26:0/2040870113 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f663c0b8f30 0x7f663c0b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:34.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.792+0000 7f6638a00640 1 -- 172.21.15.26:0/2040870113 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 529603098 0 0) 0x7f663c0d0130 con 0x7f663c15f1c0 2024-04-26T21:08:34.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.792+0000 7f6638a00640 1 -- 172.21.15.26:0/2040870113 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f661c003520 con 0x7f663c15f1c0 2024-04-26T21:08:34.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.792+0000 7f663a800640 1 -- 172.21.15.26:0/2040870113 >> v1:172.21.15.86:6789/0 conn(0x7f663c15f5a0 legacy=0x7f663c0d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:34.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.792+0000 7f663a800640 1 -- 172.21.15.26:0/2040870113 >> v1:172.21.15.26:6789/0 conn(0x7f663c15f1c0 legacy=0x7f663c0d3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:34.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.792+0000 7f663a800640 1 -- 172.21.15.26:0/2040870113 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663c0d09d0 con 0x7f663c0b8f30 2024-04-26T21:08:34.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.793+0000 7f663a800640 1 --2- 172.21.15.26:0/2040870113 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f663c0b8f30 0x7f663c0b5ce0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f66340097d0 tx=0x7f663402a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e00069557608a979 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:34.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.794+0000 7f6638a00640 1 -- 172.21.15.26:0/2040870113 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6634037070 con 0x7f663c0b8f30 2024-04-26T21:08:34.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.794+0000 7f6638a00640 1 -- 172.21.15.26:0/2040870113 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6634032440 con 0x7f663c0b8f30 2024-04-26T21:08:34.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.794+0000 7f6638a00640 1 -- 172.21.15.26:0/2040870113 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6634032810 con 0x7f663c0b8f30 2024-04-26T21:08:34.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.795+0000 7f663b200640 1 -- 172.21.15.26:0/2040870113 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f663c0b8f30 msgr2=0x7f663c0b5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:34.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.795+0000 7f663b200640 1 --2- 172.21.15.26:0/2040870113 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f663c0b8f30 0x7f663c0b5ce0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f66340097d0 tx=0x7f663402a200 comp rx=0 tx=0).stop 2024-04-26T21:08:34.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.795+0000 7f663b200640 1 -- 172.21.15.26:0/2040870113 shutdown_connections 2024-04-26T21:08:34.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.795+0000 7f663b200640 1 --2- 172.21.15.26:0/2040870113 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f663c0b8f30 0x7f663c0b5ce0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.795+0000 7f663b200640 1 -- 172.21.15.26:0/2040870113 >> 172.21.15.26:0/2040870113 conn(0x7f663c09e3b0 msgr2=0x7f663c0d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:34.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.795+0000 7f663b200640 1 -- 172.21.15.26:0/2040870113 shutdown_connections 2024-04-26T21:08:34.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.795+0000 7f663b200640 1 -- 172.21.15.26:0/2040870113 wait complete. 2024-04-26T21:08:34.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.797+0000 7f663b200640 1 Processor -- start 2024-04-26T21:08:34.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.797+0000 7f663b200640 1 -- start start 2024-04-26T21:08:34.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.798+0000 7f663b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f663c0b8f30 0x7f663c0c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:34.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.798+0000 7f663b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f663c15f1c0 0x7f663c1589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:34.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.798+0000 7f663b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f663c15f5a0 0x7f663c15a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:34.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.798+0000 7f663b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f663c0d2140 con 0x7f663c15f1c0 2024-04-26T21:08:34.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.798+0000 7f663b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f663c0d2ec0 con 0x7f663c15f5a0 2024-04-26T21:08:34.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.798+0000 7f663b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f663c0d22c0 con 0x7f663c0b8f30 2024-04-26T21:08:34.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.798+0000 7f6639400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f663c15f1c0 0x7f663c1589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:34.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.798+0000 7f6639400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f663c15f1c0 0x7f663c1589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42188/0 (socket says 172.21.15.26:42188) 2024-04-26T21:08:34.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.798+0000 7f6639400640 1 -- 172.21.15.26:0/3727012964 learned_addr learned my addr 172.21.15.26:0/3727012964 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:34.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.798+0000 7f6639e00640 1 --2- 172.21.15.26:0/3727012964 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f663c0b8f30 0x7f663c0c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:34.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.798+0000 7f663a800640 1 --2- 172.21.15.26:0/3727012964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f663c15f5a0 0x7f663c15a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:34.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.799+0000 7f6639400640 1 -- 172.21.15.26:0/3727012964 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f663c0b8f30 msgr2=0x7f663c0c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:34.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.799+0000 7f6639400640 1 --2- 172.21.15.26:0/3727012964 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f663c0b8f30 0x7f663c0c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.799+0000 7f6639400640 1 -- 172.21.15.26:0/3727012964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f663c15f5a0 msgr2=0x7f663c15a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:34.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.799+0000 7f6639400640 1 --2- 172.21.15.26:0/3727012964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f663c15f5a0 0x7f663c15a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:34.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.799+0000 7f6639400640 1 -- 172.21.15.26:0/3727012964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663c0ca210 con 0x7f663c15f1c0 2024-04-26T21:08:34.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.799+0000 7f663a800640 1 --2- 172.21.15.26:0/3727012964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f663c15f5a0 0x7f663c15a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:34.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.799+0000 7f6639400640 1 --2- 172.21.15.26:0/3727012964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f663c15f1c0 0x7f663c1589c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f662400c730 tx=0x7f662400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:34.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.799+0000 7f6632000640 1 -- 172.21.15.26:0/3727012964 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6624016020 con 0x7f663c15f1c0 2024-04-26T21:08:34.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.799+0000 7f6632000640 1 -- 172.21.15.26:0/3727012964 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6624006d50 con 0x7f663c15f1c0 2024-04-26T21:08:34.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.799+0000 7f6632000640 1 -- 172.21.15.26:0/3727012964 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6624003bb0 con 0x7f663c15f1c0 2024-04-26T21:08:34.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.799+0000 7f663b200640 1 -- 172.21.15.26:0/3727012964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663c0ca4f0 con 0x7f663c15f1c0 2024-04-26T21:08:34.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.800+0000 7f663b200640 1 -- 172.21.15.26:0/3727012964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f663c0caab0 con 0x7f663c15f1c0 2024-04-26T21:08:34.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.801+0000 7f663b200640 1 -- 172.21.15.26:0/3727012964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f663c0693d0 con 0x7f663c15f1c0 2024-04-26T21:08:34.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.802+0000 7f6632000640 1 -- 172.21.15.26:0/3727012964 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f662400a850 con 0x7f663c15f1c0 2024-04-26T21:08:34.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.803+0000 7f6632000640 1 --2- 172.21.15.26:0/3727012964 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f661806c860 0x7f661806ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:34.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.803+0000 7f6632000640 1 -- 172.21.15.26:0/3727012964 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f662408b3d0 con 0x7f663c15f1c0 2024-04-26T21:08:34.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.803+0000 7f6639e00640 1 --2- 172.21.15.26:0/3727012964 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f661806c860 0x7f661806ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:34.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.804+0000 7f6639e00640 1 --2- 172.21.15.26:0/3727012964 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f661806c860 0x7f661806ed20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f662c005880 tx=0x7f662c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:34.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:34.808+0000 7f6632000640 1 -- 172.21.15.26:0/3727012964 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6624020020 con 0x7f663c15f1c0 2024-04-26T21:08:35.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.009+0000 7f663b200640 1 -- 172.21.15.26:0/3727012964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f663c09e8f0 con 0x7f663c15f1c0 2024-04-26T21:08:35.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.010+0000 7f6632000640 1 -- 172.21.15.26:0/3727012964 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f6624019310 con 0x7f663c15f1c0 2024-04-26T21:08:35.011 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:35.011 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-04-26T21:08:35.011 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:08:35.011 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-04-26T21:08:35.011 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-04-26T21:08:35.011 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:08:35.011 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-04-26T21:08:35.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.012+0000 7f663b200640 1 -- 172.21.15.26:0/3727012964 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f661806c860 msgr2=0x7f661806ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:35.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.012+0000 7f663b200640 1 --2- 172.21.15.26:0/3727012964 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f661806c860 0x7f661806ed20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f662c005880 tx=0x7f662c009040 comp rx=0 tx=0).stop 2024-04-26T21:08:35.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.012+0000 7f663b200640 1 -- 172.21.15.26:0/3727012964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f663c15f1c0 msgr2=0x7f663c1589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:35.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.012+0000 7f663b200640 1 --2- 172.21.15.26:0/3727012964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f663c15f1c0 0x7f663c1589c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f662400c730 tx=0x7f662400cbf0 comp rx=0 tx=0).stop 2024-04-26T21:08:35.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.012+0000 7f663b200640 1 -- 172.21.15.26:0/3727012964 shutdown_connections 2024-04-26T21:08:35.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.012+0000 7f663b200640 1 --2- 172.21.15.26:0/3727012964 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f661806c860 0x7f661806ed20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:35.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.012+0000 7f663b200640 1 --2- 172.21.15.26:0/3727012964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f663c15f5a0 0x7f663c15a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:35.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.012+0000 7f663b200640 1 --2- 172.21.15.26:0/3727012964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f663c15f1c0 0x7f663c1589c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:35.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.012+0000 7f663b200640 1 --2- 172.21.15.26:0/3727012964 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f663c0b8f30 0x7f663c0c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:35.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.012+0000 7f663b200640 1 -- 172.21.15.26:0/3727012964 >> 172.21.15.26:0/3727012964 conn(0x7f663c09e3b0 msgr2=0x7f663c0b7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:35.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.012+0000 7f663b200640 1 -- 172.21.15.26:0/3727012964 shutdown_connections 2024-04-26T21:08:35.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.013+0000 7f663b200640 1 -- 172.21.15.26:0/3727012964 wait complete. 2024-04-26T21:08:35.031 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:08:35.031 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:08:35.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.244+0000 7face3400640 1 Processor -- start 2024-04-26T21:08:35.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.245+0000 7face3400640 1 -- start start 2024-04-26T21:08:35.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.246+0000 7face3400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7facdc15f440 0x7facdc0d5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:35.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.246+0000 7face3400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facdc0d2550 con 0x7facdc0d3060 2024-04-26T21:08:35.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.246+0000 7face3400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facdc0d2830 con 0x7facdc0d3e90 2024-04-26T21:08:35.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.246+0000 7face3400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7facdc0b4ab0 con 0x7facdc15f440 2024-04-26T21:08:35.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.246+0000 7face0a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7facdc15f440 0x7facdc0d5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:35.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.247+0000 7face0a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7facdc15f440 0x7facdc0d5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42204/0 (socket says 172.21.15.26:42204) 2024-04-26T21:08:35.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.247+0000 7face0a00640 1 -- 172.21.15.26:0/1601254744 learned_addr learned my addr 172.21.15.26:0/1601254744 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:35.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.247+0000 7face0a00640 1 -- 172.21.15.26:0/1601254744 >> v1:172.21.15.86:6789/0 conn(0x7facdc0d3e90 legacy=0x7facdc0b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:35.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.247+0000 7face0a00640 1 -- 172.21.15.26:0/1601254744 >> v1:172.21.15.26:6789/0 conn(0x7facdc0d3060 legacy=0x7facdc0d3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:35.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.247+0000 7face0a00640 1 -- 172.21.15.26:0/1601254744 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc0b4740 con 0x7facdc15f440 2024-04-26T21:08:35.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.248+0000 7face0a00640 1 --2- 172.21.15.26:0/1601254744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facdc15f440 0x7facdc0d5cc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7facdc0b4c30 tx=0x7facd002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc085fb3e38eaf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:35.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.248+0000 7facdb400640 1 -- 172.21.15.26:0/1601254744 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd0037070 con 0x7facdc15f440 2024-04-26T21:08:35.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.248+0000 7facdb400640 1 -- 172.21.15.26:0/1601254744 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd0032440 con 0x7facdc15f440 2024-04-26T21:08:35.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.248+0000 7facdb400640 1 -- 172.21.15.26:0/1601254744 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd0032810 con 0x7facdc15f440 2024-04-26T21:08:35.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.249+0000 7face3400640 1 -- 172.21.15.26:0/1601254744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facdc15f440 msgr2=0x7facdc0d5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:35.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.249+0000 7face3400640 1 --2- 172.21.15.26:0/1601254744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facdc15f440 0x7facdc0d5cc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7facdc0b4c30 tx=0x7facd002a210 comp rx=0 tx=0).stop 2024-04-26T21:08:35.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.249+0000 7face3400640 1 -- 172.21.15.26:0/1601254744 shutdown_connections 2024-04-26T21:08:35.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.249+0000 7face3400640 1 --2- 172.21.15.26:0/1601254744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facdc15f440 0x7facdc0d5cc0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:35.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.249+0000 7face3400640 1 -- 172.21.15.26:0/1601254744 >> 172.21.15.26:0/1601254744 conn(0x7facdc0d51a0 msgr2=0x7facdc0d55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:35.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.249+0000 7face3400640 1 -- 172.21.15.26:0/1601254744 shutdown_connections 2024-04-26T21:08:35.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.249+0000 7face3400640 1 -- 172.21.15.26:0/1601254744 wait complete. 2024-04-26T21:08:35.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.251+0000 7face3400640 1 Processor -- start 2024-04-26T21:08:35.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.251+0000 7face3400640 1 -- start start 2024-04-26T21:08:35.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.252+0000 7face3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facdc0d3060 0x7facdc156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:35.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.252+0000 7face3400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facdc0d3e90 0x7facdc147d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:35.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.252+0000 7face3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facdc15f440 0x7facdc06de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:35.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.252+0000 7face3400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7facdc15f820 con 0x7facdc0d3060 2024-04-26T21:08:35.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.252+0000 7face3400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7facdc0d2b10 con 0x7facdc15f440 2024-04-26T21:08:35.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.252+0000 7face3400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7facdc15f9a0 con 0x7facdc0d3e90 2024-04-26T21:08:35.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.252+0000 7face0a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facdc0d3060 0x7facdc156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:35.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.252+0000 7face0a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facdc0d3060 0x7facdc156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42212/0 (socket says 172.21.15.26:42212) 2024-04-26T21:08:35.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.252+0000 7face0a00640 1 -- 172.21.15.26:0/2529673331 learned_addr learned my addr 172.21.15.26:0/2529673331 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:35.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.253+0000 7face1400640 1 --2- 172.21.15.26:0/2529673331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facdc15f440 0x7facdc06de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:35.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.253+0000 7facdbe00640 1 --2- 172.21.15.26:0/2529673331 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facdc0d3e90 0x7facdc147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:35.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.253+0000 7face0a00640 1 -- 172.21.15.26:0/2529673331 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facdc0d3e90 msgr2=0x7facdc147d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:35.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.253+0000 7face0a00640 1 --2- 172.21.15.26:0/2529673331 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facdc0d3e90 0x7facdc147d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:35.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.253+0000 7face0a00640 1 -- 172.21.15.26:0/2529673331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facdc15f440 msgr2=0x7facdc06de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:35.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.253+0000 7face0a00640 1 --2- 172.21.15.26:0/2529673331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facdc15f440 0x7facdc06de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:35.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.253+0000 7face0a00640 1 -- 172.21.15.26:0/2529673331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc1482a0 con 0x7facdc0d3060 2024-04-26T21:08:35.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.253+0000 7face1400640 1 --2- 172.21.15.26:0/2529673331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facdc15f440 0x7facdc06de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:35.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.253+0000 7face0a00640 1 --2- 172.21.15.26:0/2529673331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facdc0d3060 0x7facdc156c00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7facd0000c00 tx=0x7facd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:35.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.253+0000 7facd8c00640 1 -- 172.21.15.26:0/2529673331 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd0041020 con 0x7facdc0d3060 2024-04-26T21:08:35.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.253+0000 7facd8c00640 1 -- 172.21.15.26:0/2529673331 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd0030970 con 0x7facdc0d3060 2024-04-26T21:08:35.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.253+0000 7facd8c00640 1 -- 172.21.15.26:0/2529673331 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd0037040 con 0x7facdc0d3060 2024-04-26T21:08:35.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.253+0000 7face3400640 1 -- 172.21.15.26:0/2529673331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facdc162b20 con 0x7facdc0d3060 2024-04-26T21:08:35.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.253+0000 7face3400640 1 -- 172.21.15.26:0/2529673331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facdc161d60 con 0x7facdc0d3060 2024-04-26T21:08:35.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.254+0000 7face3400640 1 -- 172.21.15.26:0/2529673331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facdc06a700 con 0x7facdc0d3060 2024-04-26T21:08:35.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.256+0000 7facd8c00640 1 -- 172.21.15.26:0/2529673331 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7facd0044020 con 0x7facdc0d3060 2024-04-26T21:08:35.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.257+0000 7facd8c00640 1 --2- 172.21.15.26:0/2529673331 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7facbc06c890 0x7facbc06ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:35.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.257+0000 7facd8c00640 1 -- 172.21.15.26:0/2529673331 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7facd00ac0e0 con 0x7facdc0d3060 2024-04-26T21:08:35.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.258+0000 7facdbe00640 1 --2- 172.21.15.26:0/2529673331 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7facbc06c890 0x7facbc06ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:35.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.258+0000 7facdbe00640 1 --2- 172.21.15.26:0/2529673331 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7facbc06c890 0x7facbc06ed50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7facdc1614a0 tx=0x7facc8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:35.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.262+0000 7facd8c00640 1 -- 172.21.15.26:0/2529673331 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7facd004b030 con 0x7facdc0d3060 2024-04-26T21:08:35.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:35.466+0000 7face3400640 1 -- 172.21.15.26:0/2529673331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7facdc0cf560 con 0x7facdc0d3060 2024-04-26T21:08:36.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.123+0000 7facd8c00640 1 -- 172.21.15.26:0/2529673331 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7facdc1482a0 con 0x7facdc0d3060 2024-04-26T21:08:36.124 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-04-26T21:08:36.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.127+0000 7face3400640 1 -- 172.21.15.26:0/2529673331 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7facbc06c890 msgr2=0x7facbc06ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:36.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.127+0000 7face3400640 1 --2- 172.21.15.26:0/2529673331 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7facbc06c890 0x7facbc06ed50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7facdc1614a0 tx=0x7facc8009040 comp rx=0 tx=0).stop 2024-04-26T21:08:36.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.127+0000 7face3400640 1 -- 172.21.15.26:0/2529673331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facdc0d3060 msgr2=0x7facdc156c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:36.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.127+0000 7face3400640 1 --2- 172.21.15.26:0/2529673331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facdc0d3060 0x7facdc156c00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7facd0000c00 tx=0x7facd00046c0 comp rx=0 tx=0).stop 2024-04-26T21:08:36.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.127+0000 7face3400640 1 -- 172.21.15.26:0/2529673331 shutdown_connections 2024-04-26T21:08:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.127+0000 7face3400640 1 --2- 172.21.15.26:0/2529673331 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7facbc06c890 0x7facbc06ed50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.127+0000 7face3400640 1 --2- 172.21.15.26:0/2529673331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facdc15f440 0x7facdc06de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.127+0000 7face3400640 1 --2- 172.21.15.26:0/2529673331 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facdc0d3e90 0x7facdc147d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.127+0000 7face3400640 1 --2- 172.21.15.26:0/2529673331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facdc0d3060 0x7facdc156c00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.127+0000 7face3400640 1 -- 172.21.15.26:0/2529673331 >> 172.21.15.26:0/2529673331 conn(0x7facdc0d51a0 msgr2=0x7facdc0b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.128+0000 7face3400640 1 -- 172.21.15.26:0/2529673331 shutdown_connections 2024-04-26T21:08:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.128+0000 7face3400640 1 -- 172.21.15.26:0/2529673331 wait complete. 2024-04-26T21:08:36.163 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:08:36.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.364+0000 7fd481200640 1 Processor -- start 2024-04-26T21:08:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.364+0000 7fd481200640 1 -- start start 2024-04-26T21:08:36.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.365+0000 7fd481200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd47c15bdc0 0x7fd47c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:36.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.365+0000 7fd481200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd47c073530 0x7fd47c0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:36.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.365+0000 7fd481200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd47c06fbb0 con 0x7fd47c06e210 2024-04-26T21:08:36.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.365+0000 7fd481200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fd47c0b4cf0 con 0x7fd47c073530 2024-04-26T21:08:36.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.365+0000 7fd481200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd47c0cf280 con 0x7fd47c15bdc0 2024-04-26T21:08:36.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.366+0000 7fd47be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd47c073530 0x7fd47c0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:36.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.366+0000 7fd47be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd47c073530 0x7fd47c0cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42216/0 (socket says 172.21.15.26:42216) 2024-04-26T21:08:36.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.366+0000 7fd47be00640 1 -- 172.21.15.26:0/3084669310 learned_addr learned my addr 172.21.15.26:0/3084669310 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:36.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.366+0000 7fd47b400640 1 --2- 172.21.15.26:0/3084669310 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd47c15bdc0 0x7fd47c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:36.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.366+0000 7fd47be00640 1 -- 172.21.15.26:0/3084669310 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd47c15bdc0 msgr2=0x7fd47c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:36.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.366+0000 7fd47be00640 1 --2- 172.21.15.26:0/3084669310 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd47c15bdc0 0x7fd47c0b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:36.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.366+0000 7fd47be00640 1 -- 172.21.15.26:0/3084669310 >> v1:172.21.15.86:6789/0 conn(0x7fd47c06e210 legacy=0x7fd47c072e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:36.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.366+0000 7fd47be00640 1 -- 172.21.15.26:0/3084669310 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47c06e780 con 0x7fd47c073530 2024-04-26T21:08:36.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.366+0000 7fd47b400640 1 --2- 172.21.15.26:0/3084669310 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd47c15bdc0 0x7fd47c0b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:36.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.367+0000 7fd47be00640 1 --2- 172.21.15.26:0/3084669310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd47c073530 0x7fd47c0cf710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd474009e50 tx=0x7fd47402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a5b951fdc01f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:36.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.368+0000 7fd47aa00640 1 -- 172.21.15.26:0/3084669310 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd474036070 con 0x7fd47c073530 2024-04-26T21:08:36.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.368+0000 7fd47aa00640 1 -- 172.21.15.26:0/3084669310 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd474031600 con 0x7fd47c073530 2024-04-26T21:08:36.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.368+0000 7fd47aa00640 1 -- 172.21.15.26:0/3084669310 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4740319d0 con 0x7fd47c073530 2024-04-26T21:08:36.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.368+0000 7fd481200640 1 -- 172.21.15.26:0/3084669310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd47c073530 msgr2=0x7fd47c0cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:36.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.368+0000 7fd481200640 1 --2- 172.21.15.26:0/3084669310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd47c073530 0x7fd47c0cf710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd474009e50 tx=0x7fd47402a630 comp rx=0 tx=0).stop 2024-04-26T21:08:36.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.368+0000 7fd481200640 1 -- 172.21.15.26:0/3084669310 shutdown_connections 2024-04-26T21:08:36.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.368+0000 7fd481200640 1 --2- 172.21.15.26:0/3084669310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd47c073530 0x7fd47c0cf710 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:36.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.368+0000 7fd481200640 1 --2- 172.21.15.26:0/3084669310 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd47c15bdc0 0x7fd47c0b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:36.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.368+0000 7fd481200640 1 -- 172.21.15.26:0/3084669310 >> 172.21.15.26:0/3084669310 conn(0x7fd47c0b2a10 msgr2=0x7fd47c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:36.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.369+0000 7fd481200640 1 -- 172.21.15.26:0/3084669310 shutdown_connections 2024-04-26T21:08:36.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.369+0000 7fd481200640 1 -- 172.21.15.26:0/3084669310 wait complete. 2024-04-26T21:08:36.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.370+0000 7fd481200640 1 Processor -- start 2024-04-26T21:08:36.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.370+0000 7fd481200640 1 -- start start 2024-04-26T21:08:36.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.371+0000 7fd481200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd47c06e210 0x7fd47c14d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:36.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.371+0000 7fd481200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd47c073530 0x7fd47c14dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:36.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.371+0000 7fd481200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd47c15bdc0 0x7fd47c147b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:36.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.371+0000 7fd481200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd47c072380 con 0x7fd47c06e210 2024-04-26T21:08:36.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.371+0000 7fd481200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd47c070130 con 0x7fd47c15bdc0 2024-04-26T21:08:36.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.371+0000 7fd481200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd47c15ad20 con 0x7fd47c073530 2024-04-26T21:08:36.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.372+0000 7fd47b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd47c06e210 0x7fd47c14d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:36.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.372+0000 7fd47b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd47c06e210 0x7fd47c14d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42232/0 (socket says 172.21.15.26:42232) 2024-04-26T21:08:36.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.372+0000 7fd47b400640 1 -- 172.21.15.26:0/1772231376 learned_addr learned my addr 172.21.15.26:0/1772231376 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:36.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.372+0000 7fd47be00640 1 --2- 172.21.15.26:0/1772231376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd47c15bdc0 0x7fd47c147b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:36.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.372+0000 7fd473e00640 1 --2- 172.21.15.26:0/1772231376 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd47c073530 0x7fd47c14dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:36.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.372+0000 7fd47b400640 1 -- 172.21.15.26:0/1772231376 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd47c073530 msgr2=0x7fd47c14dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:36.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.372+0000 7fd47b400640 1 --2- 172.21.15.26:0/1772231376 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd47c073530 0x7fd47c14dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:36.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.372+0000 7fd47b400640 1 -- 172.21.15.26:0/1772231376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd47c15bdc0 msgr2=0x7fd47c147b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:36.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.372+0000 7fd47b400640 1 --2- 172.21.15.26:0/1772231376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd47c15bdc0 0x7fd47c147b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:36.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.372+0000 7fd47b400640 1 -- 172.21.15.26:0/1772231376 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47c15ce50 con 0x7fd47c06e210 2024-04-26T21:08:36.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.372+0000 7fd473e00640 1 --2- 172.21.15.26:0/1772231376 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd47c073530 0x7fd47c14dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:36.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.372+0000 7fd47b400640 1 --2- 172.21.15.26:0/1772231376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd47c06e210 0x7fd47c14d6a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00c7f0 tx=0x7fd46c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:36.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.373+0000 7fd473400640 1 -- 172.21.15.26:0/1772231376 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c015070 con 0x7fd47c06e210 2024-04-26T21:08:36.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.373+0000 7fd473400640 1 -- 172.21.15.26:0/1772231376 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd46c007820 con 0x7fd47c06e210 2024-04-26T21:08:36.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.373+0000 7fd473400640 1 -- 172.21.15.26:0/1772231376 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c007bd0 con 0x7fd47c06e210 2024-04-26T21:08:36.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.373+0000 7fd481200640 1 -- 172.21.15.26:0/1772231376 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd47c15d130 con 0x7fd47c06e210 2024-04-26T21:08:36.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.373+0000 7fd481200640 1 -- 172.21.15.26:0/1772231376 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd47c15d6f0 con 0x7fd47c06e210 2024-04-26T21:08:36.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.374+0000 7fd481200640 1 -- 172.21.15.26:0/1772231376 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd47c06a700 con 0x7fd47c06e210 2024-04-26T21:08:36.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.375+0000 7fd473400640 1 -- 172.21.15.26:0/1772231376 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd46c018070 con 0x7fd47c06e210 2024-04-26T21:08:36.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.376+0000 7fd473400640 1 --2- 172.21.15.26:0/1772231376 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd45806c8f0 0x7fd45806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:36.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.376+0000 7fd473400640 1 -- 172.21.15.26:0/1772231376 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7fd46c08ba80 con 0x7fd47c06e210 2024-04-26T21:08:36.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.376+0000 7fd473e00640 1 --2- 172.21.15.26:0/1772231376 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd45806c8f0 0x7fd45806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:36.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.377+0000 7fd473e00640 1 --2- 172.21.15.26:0/1772231376 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd45806c8f0 0x7fd45806edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd47c1f6700 tx=0x7fd464009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:36.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.382+0000 7fd473400640 1 -- 172.21.15.26:0/1772231376 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd46c05a090 con 0x7fd47c06e210 2024-04-26T21:08:36.546 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:08:36.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:36.592+0000 7fd481200640 1 -- 172.21.15.26:0/1772231376 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd47c0d0610 con 0x7fd47c06e210 2024-04-26T21:08:36.610 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:08:37.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.130+0000 7fd473400640 1 -- 172.21.15.26:0/1772231376 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7fd46c059990 con 0x7fd47c06e210 2024-04-26T21:08:37.131 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-04-26T21:08:37.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.134+0000 7fd481200640 1 -- 172.21.15.26:0/1772231376 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd45806c8f0 msgr2=0x7fd45806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:37.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.134+0000 7fd481200640 1 --2- 172.21.15.26:0/1772231376 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd45806c8f0 0x7fd45806edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd47c1f6700 tx=0x7fd464009040 comp rx=0 tx=0).stop 2024-04-26T21:08:37.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.134+0000 7fd481200640 1 -- 172.21.15.26:0/1772231376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd47c06e210 msgr2=0x7fd47c14d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:37.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.134+0000 7fd481200640 1 --2- 172.21.15.26:0/1772231376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd47c06e210 0x7fd47c14d6a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00c7f0 tx=0x7fd46c00ccb0 comp rx=0 tx=0).stop 2024-04-26T21:08:37.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.135+0000 7fd481200640 1 -- 172.21.15.26:0/1772231376 shutdown_connections 2024-04-26T21:08:37.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.135+0000 7fd481200640 1 --2- 172.21.15.26:0/1772231376 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd45806c8f0 0x7fd45806edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:37.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.135+0000 7fd481200640 1 --2- 172.21.15.26:0/1772231376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd47c15bdc0 0x7fd47c147b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:37.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.135+0000 7fd481200640 1 --2- 172.21.15.26:0/1772231376 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd47c073530 0x7fd47c14dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:37.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.135+0000 7fd481200640 1 --2- 172.21.15.26:0/1772231376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd47c06e210 0x7fd47c14d6a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:37.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.135+0000 7fd481200640 1 -- 172.21.15.26:0/1772231376 >> 172.21.15.26:0/1772231376 conn(0x7fd47c0b2a10 msgr2=0x7fd47c0b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:37.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.135+0000 7fd481200640 1 -- 172.21.15.26:0/1772231376 shutdown_connections 2024-04-26T21:08:37.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.135+0000 7fd481200640 1 -- 172.21.15.26:0/1772231376 wait complete. 2024-04-26T21:08:37.172 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:08:37.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.353+0000 7f4c6b400640 1 Processor -- start 2024-04-26T21:08:37.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.353+0000 7f4c6b400640 1 -- start start 2024-04-26T21:08:37.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.355+0000 7f4c6b400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c6406f8c0 0x7f4c640b6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:37.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.355+0000 7f4c6b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4c640b6650 0x7f4c64166c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:37.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.355+0000 7f4c6b400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c64146c00 con 0x7f4c640b5d30 2024-04-26T21:08:37.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.355+0000 7f4c6b400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4c64073700 con 0x7f4c640b6650 2024-04-26T21:08:37.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.355+0000 7f4c6b400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4c6406e420 con 0x7f4c6406f8c0 2024-04-26T21:08:37.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.356+0000 7f4c63e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c6406f8c0 0x7f4c640b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:37.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.356+0000 7f4c63e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c6406f8c0 0x7f4c640b6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.26:32884/0 (socket says 172.21.15.26:32884) 2024-04-26T21:08:37.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.356+0000 7f4c63e00640 1 -- 172.21.15.26:0/3113286410 learned_addr learned my addr 172.21.15.26:0/3113286410 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:37.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.356+0000 7f4c69400640 1 --2- 172.21.15.26:0/3113286410 >> v2:172.21.15.86:3300/0 conn(0x7f4c640b6650 0x7f4c64166c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:37.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.356+0000 7f4c68a00640 1 --1- 172.21.15.26:0/3113286410 >> v1:172.21.15.86:6789/0 conn(0x7f4c640b5d30 0x7f4c640b3d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.26:54016/0 (socket says 172.21.15.26:54016) 2024-04-26T21:08:37.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.357+0000 7f4c63e00640 1 -- 172.21.15.26:0/3113286410 >> v2:172.21.15.86:3300/0 conn(0x7f4c640b6650 msgr2=0x7f4c64166c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:37.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.357+0000 7f4c63e00640 1 --2- 172.21.15.26:0/3113286410 >> v2:172.21.15.86:3300/0 conn(0x7f4c640b6650 0x7f4c64166c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:37.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.357+0000 7f4c63e00640 1 -- 172.21.15.26:0/3113286410 >> v1:172.21.15.86:6789/0 conn(0x7f4c640b5d30 legacy=0x7f4c640b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:37.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.357+0000 7f4c63e00640 1 -- 172.21.15.26:0/3113286410 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c64072e60 con 0x7f4c6406f8c0 2024-04-26T21:08:37.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.358+0000 7f4c63e00640 1 --2- 172.21.15.26:0/3113286410 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c6406f8c0 0x7f4c640b6110 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009870 tx=0x7f4c5002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e73eb0a26c85f5cf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:37.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.358+0000 7f4c63400640 1 -- 172.21.15.26:0/3113286410 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50037070 con 0x7f4c6406f8c0 2024-04-26T21:08:37.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.358+0000 7f4c63400640 1 -- 172.21.15.26:0/3113286410 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c50032440 con 0x7f4c6406f8c0 2024-04-26T21:08:37.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.358+0000 7f4c63400640 1 -- 172.21.15.26:0/3113286410 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50032810 con 0x7f4c6406f8c0 2024-04-26T21:08:37.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.359+0000 7f4c6b400640 1 -- 172.21.15.26:0/3113286410 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c6406f8c0 msgr2=0x7f4c640b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:37.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.359+0000 7f4c6b400640 1 --2- 172.21.15.26:0/3113286410 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c6406f8c0 0x7f4c640b6110 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009870 tx=0x7f4c5002a270 comp rx=0 tx=0).stop 2024-04-26T21:08:37.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.359+0000 7f4c6b400640 1 -- 172.21.15.26:0/3113286410 shutdown_connections 2024-04-26T21:08:37.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.359+0000 7f4c6b400640 1 --2- 172.21.15.26:0/3113286410 >> v2:172.21.15.86:3300/0 conn(0x7f4c640b6650 0x7f4c64166c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:37.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.359+0000 7f4c6b400640 1 --2- 172.21.15.26:0/3113286410 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c6406f8c0 0x7f4c640b6110 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:37.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.359+0000 7f4c6b400640 1 -- 172.21.15.26:0/3113286410 >> 172.21.15.26:0/3113286410 conn(0x7f4c640b2a80 msgr2=0x7f4c640b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:37.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.359+0000 7f4c6b400640 1 -- 172.21.15.26:0/3113286410 shutdown_connections 2024-04-26T21:08:37.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.359+0000 7f4c6b400640 1 -- 172.21.15.26:0/3113286410 wait complete. 2024-04-26T21:08:37.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.362+0000 7f4c6b400640 1 Processor -- start 2024-04-26T21:08:37.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.362+0000 7f4c6b400640 1 -- start start 2024-04-26T21:08:37.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.363+0000 7f4c6b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c6406f8c0 0x7f4c640d0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:37.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.363+0000 7f4c6b400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c640b5d30 0x7f4c641e7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:37.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.363+0000 7f4c6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c640b6650 0x7f4c641e7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:37.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.363+0000 7f4c6b400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4c640b4490 con 0x7f4c6406f8c0 2024-04-26T21:08:37.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.363+0000 7f4c6b400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c64073420 con 0x7f4c640b6650 2024-04-26T21:08:37.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.363+0000 7f4c6b400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4c64072a80 con 0x7f4c640b5d30 2024-04-26T21:08:37.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.364+0000 7f4c68a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c6406f8c0 0x7f4c640d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:37.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.364+0000 7f4c68a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c6406f8c0 0x7f4c640d0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42246/0 (socket says 172.21.15.26:42246) 2024-04-26T21:08:37.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.364+0000 7f4c68a00640 1 -- 172.21.15.26:0/1328339022 learned_addr learned my addr 172.21.15.26:0/1328339022 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:37.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.364+0000 7f4c69400640 1 --2- 172.21.15.26:0/1328339022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c640b6650 0x7f4c641e7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:37.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.364+0000 7f4c63e00640 1 --2- 172.21.15.26:0/1328339022 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c640b5d30 0x7f4c641e7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:37.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.364+0000 7f4c68a00640 1 -- 172.21.15.26:0/1328339022 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c640b5d30 msgr2=0x7f4c641e7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:37.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.364+0000 7f4c68a00640 1 --2- 172.21.15.26:0/1328339022 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c640b5d30 0x7f4c641e7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:37.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.364+0000 7f4c68a00640 1 -- 172.21.15.26:0/1328339022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c640b6650 msgr2=0x7f4c641e7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:37.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.364+0000 7f4c68a00640 1 --2- 172.21.15.26:0/1328339022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c640b6650 0x7f4c641e7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:37.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.364+0000 7f4c68a00640 1 -- 172.21.15.26:0/1328339022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c641f8550 con 0x7f4c6406f8c0 2024-04-26T21:08:37.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.365+0000 7f4c68a00640 1 --2- 172.21.15.26:0/1328339022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c6406f8c0 0x7f4c640d0330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800c500 tx=0x7f4c5800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:37.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.365+0000 7f4c60c00640 1 -- 172.21.15.26:0/1328339022 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58018070 con 0x7f4c6406f8c0 2024-04-26T21:08:37.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.365+0000 7f4c60c00640 1 -- 172.21.15.26:0/1328339022 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c58007500 con 0x7f4c6406f8c0 2024-04-26T21:08:37.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.365+0000 7f4c6b400640 1 -- 172.21.15.26:0/1328339022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c641f8830 con 0x7f4c6406f8c0 2024-04-26T21:08:37.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.365+0000 7f4c60c00640 1 -- 172.21.15.26:0/1328339022 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c580049e0 con 0x7f4c6406f8c0 2024-04-26T21:08:37.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.365+0000 7f4c6b400640 1 -- 172.21.15.26:0/1328339022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c641f8df0 con 0x7f4c6406f8c0 2024-04-26T21:08:37.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.366+0000 7f4c6b400640 1 -- 172.21.15.26:0/1328339022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c6406a700 con 0x7f4c6406f8c0 2024-04-26T21:08:37.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.368+0000 7f4c60c00640 1 -- 172.21.15.26:0/1328339022 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c5801b070 con 0x7f4c6406f8c0 2024-04-26T21:08:37.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.369+0000 7f4c60c00640 1 --2- 172.21.15.26:0/1328339022 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4c4406c8f0 0x7f4c4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:37.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.369+0000 7f4c60c00640 1 -- 172.21.15.26:0/1328339022 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f4c5808aef0 con 0x7f4c6406f8c0 2024-04-26T21:08:37.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.369+0000 7f4c63e00640 1 --2- 172.21.15.26:0/1328339022 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4c4406c8f0 0x7f4c4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:37.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.370+0000 7f4c63e00640 1 --2- 172.21.15.26:0/1328339022 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4c4406c8f0 0x7f4c4406edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4c50034000 tx=0x7f4c5002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:37.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.374+0000 7f4c60c00640 1 -- 172.21.15.26:0/1328339022 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4c58059370 con 0x7f4c6406f8c0 2024-04-26T21:08:37.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:37.621+0000 7f4c6b400640 1 -- 172.21.15.26:0/1328339022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4c64145f70 con 0x7f4c6406f8c0 2024-04-26T21:08:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.184+0000 7f4c60c00640 1 -- 172.21.15.26:0/1328339022 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f4c5805f030 con 0x7f4c6406f8c0 2024-04-26T21:08:38.190 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T21:08:38.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.193+0000 7f4c6b400640 1 -- 172.21.15.26:0/1328339022 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4c4406c8f0 msgr2=0x7f4c4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:38.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.193+0000 7f4c6b400640 1 --2- 172.21.15.26:0/1328339022 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4c4406c8f0 0x7f4c4406edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4c50034000 tx=0x7f4c5002e000 comp rx=0 tx=0).stop 2024-04-26T21:08:38.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.193+0000 7f4c6b400640 1 -- 172.21.15.26:0/1328339022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c6406f8c0 msgr2=0x7f4c640d0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:38.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.193+0000 7f4c6b400640 1 --2- 172.21.15.26:0/1328339022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c6406f8c0 0x7f4c640d0330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800c500 tx=0x7f4c5800c9c0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.194+0000 7f4c6b400640 1 -- 172.21.15.26:0/1328339022 shutdown_connections 2024-04-26T21:08:38.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.194+0000 7f4c6b400640 1 --2- 172.21.15.26:0/1328339022 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4c4406c8f0 0x7f4c4406edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.194+0000 7f4c6b400640 1 --2- 172.21.15.26:0/1328339022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c640b6650 0x7f4c641e7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.194+0000 7f4c6b400640 1 --2- 172.21.15.26:0/1328339022 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c640b5d30 0x7f4c641e7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.194+0000 7f4c6b400640 1 --2- 172.21.15.26:0/1328339022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c6406f8c0 0x7f4c640d0330 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.194+0000 7f4c6b400640 1 -- 172.21.15.26:0/1328339022 >> 172.21.15.26:0/1328339022 conn(0x7f4c640b2a80 msgr2=0x7f4c64144750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:38.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.195+0000 7f4c6b400640 1 -- 172.21.15.26:0/1328339022 shutdown_connections 2024-04-26T21:08:38.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.195+0000 7f4c6b400640 1 -- 172.21.15.26:0/1328339022 wait complete. 2024-04-26T21:08:38.232 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:08:38.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.407+0000 7fc03c000640 1 Processor -- start 2024-04-26T21:08:38.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.407+0000 7fc03c000640 1 -- start start 2024-04-26T21:08:38.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.408+0000 7fc03c000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc03415d170 0x7fc0340d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:38.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.408+0000 7fc03c000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc03415d550 con 0x7fc0340ce070 2024-04-26T21:08:38.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.409+0000 7fc03c000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0340d1210 con 0x7fc0340cd880 2024-04-26T21:08:38.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.409+0000 7fc03c000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc0340b6960 con 0x7fc03415d170 2024-04-26T21:08:38.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.409+0000 7fc03a000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc0340ce070 0x7fc03406c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35456/0 (socket says 172.21.15.26:35456) 2024-04-26T21:08:38.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.409+0000 7fc03a000640 1 -- 172.21.15.26:0/2888529030 learned_addr learned my addr 172.21.15.26:0/2888529030 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:38.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.409+0000 7fc039600640 1 --2- 172.21.15.26:0/2888529030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc03415d170 0x7fc0340d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:38.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.409+0000 7fc033e00640 1 -- 172.21.15.26:0/2888529030 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 258887012 0 0) 0x7fc03415d550 con 0x7fc0340ce070 2024-04-26T21:08:38.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.409+0000 7fc033e00640 1 -- 172.21.15.26:0/2888529030 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc01c003520 con 0x7fc0340ce070 2024-04-26T21:08:38.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.410+0000 7fc033e00640 1 -- 172.21.15.26:0/2888529030 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 164996252 0 0) 0x7fc01c003520 con 0x7fc0340ce070 2024-04-26T21:08:38.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.410+0000 7fc033e00640 1 -- 172.21.15.26:0/2888529030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc03415d170 msgr2=0x7fc0340d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:38.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.410+0000 7fc033e00640 1 --2- 172.21.15.26:0/2888529030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc03415d170 0x7fc0340d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.410+0000 7fc033e00640 1 -- 172.21.15.26:0/2888529030 >> v1:172.21.15.86:6789/0 conn(0x7fc0340cd880 legacy=0x7fc0340b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:38.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.410+0000 7fc033e00640 1 -- 172.21.15.26:0/2888529030 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0340b48e0 con 0x7fc0340ce070 2024-04-26T21:08:38.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.410+0000 7fc033e00640 1 -- 172.21.15.26:0/2888529030 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7fc02c001670 con 0x7fc0340ce070 2024-04-26T21:08:38.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.410+0000 7fc033e00640 1 -- 172.21.15.26:0/2888529030 >> v1:172.21.15.26:6789/0 conn(0x7fc0340ce070 legacy=0x7fc03406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:38.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.410+0000 7fc033e00640 1 --2- 172.21.15.26:0/2888529030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc01c0039e0 0x7fc01c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:38.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.410+0000 7fc033e00640 1 --2- 172.21.15.26:0/2888529030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc01c004800 0x7fc01c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:38.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.410+0000 7fc033e00640 1 --2- 172.21.15.26:0/2888529030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc03415d170 0x7fc01c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:38.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.410+0000 7fc033e00640 1 -- 172.21.15.26:0/2888529030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c00ba70 con 0x7fc01c004800 2024-04-26T21:08:38.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.410+0000 7fc033e00640 1 -- 172.21.15.26:0/2888529030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c00bbf0 con 0x7fc01c0039e0 2024-04-26T21:08:38.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.410+0000 7fc033e00640 1 -- 172.21.15.26:0/2888529030 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc01c00bdf0 con 0x7fc03415d170 2024-04-26T21:08:38.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.410+0000 7fc038c00640 1 --2- 172.21.15.26:0/2888529030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc01c004800 0x7fc01c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:38.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.411+0000 7fc039600640 1 --2- 172.21.15.26:0/2888529030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc01c0039e0 0x7fc01c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:38.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.411+0000 7fc03a000640 1 --2- 172.21.15.26:0/2888529030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc03415d170 0x7fc01c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:38.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.411+0000 7fc038c00640 1 -- 172.21.15.26:0/2888529030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc03415d170 msgr2=0x7fc01c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:38.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.411+0000 7fc038c00640 1 --2- 172.21.15.26:0/2888529030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc03415d170 0x7fc01c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.411+0000 7fc038c00640 1 -- 172.21.15.26:0/2888529030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc01c0039e0 msgr2=0x7fc01c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:38.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.411+0000 7fc038c00640 1 --2- 172.21.15.26:0/2888529030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc01c0039e0 0x7fc01c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.411+0000 7fc038c00640 1 -- 172.21.15.26:0/2888529030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c003520 con 0x7fc01c004800 2024-04-26T21:08:38.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.412+0000 7fc038c00640 1 --2- 172.21.15.26:0/2888529030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc01c004800 0x7fc01c004c80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc028001e20 tx=0x7fc01c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e3a158076002cf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.412+0000 7fc033e00640 1 -- 172.21.15.26:0/2888529030 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028001a70 con 0x7fc01c004800 2024-04-26T21:08:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.412+0000 7fc033e00640 1 -- 172.21.15.26:0/2888529030 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02802ae10 con 0x7fc01c004800 2024-04-26T21:08:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.412+0000 7fc033e00640 1 -- 172.21.15.26:0/2888529030 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028003bb0 con 0x7fc01c004800 2024-04-26T21:08:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.413+0000 7fc03c000640 1 -- 172.21.15.26:0/2888529030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc01c004800 msgr2=0x7fc01c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.413+0000 7fc03c000640 1 --2- 172.21.15.26:0/2888529030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc01c004800 0x7fc01c004c80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc028001e20 tx=0x7fc01c005830 comp rx=0 tx=0).stop 2024-04-26T21:08:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.413+0000 7fc03a000640 1 -- 172.21.15.26:0/2888529030 reap_dead start 2024-04-26T21:08:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.413+0000 7fc03c000640 1 -- 172.21.15.26:0/2888529030 shutdown_connections 2024-04-26T21:08:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.413+0000 7fc03c000640 1 -- 172.21.15.26:0/2888529030 >> 172.21.15.26:0/2888529030 conn(0x7fc0340d1620 msgr2=0x7fc0340d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.413+0000 7fc03c000640 1 -- 172.21.15.26:0/2888529030 shutdown_connections 2024-04-26T21:08:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.413+0000 7fc03c000640 1 -- 172.21.15.26:0/2888529030 wait complete. 2024-04-26T21:08:38.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.415+0000 7fc03c000640 1 Processor -- start 2024-04-26T21:08:38.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.415+0000 7fc03c000640 1 -- start start 2024-04-26T21:08:38.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.416+0000 7fc03c000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc01c004800 0x7fc03406ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:38.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.416+0000 7fc03c000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0341f5700 0x7fc0340b2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:38.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.416+0000 7fc03c000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc03406fa10 0x7fc03406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:38.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.416+0000 7fc03c000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc0340b5ad0 con 0x7fc01c004800 2024-04-26T21:08:38.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.416+0000 7fc03c000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0340b4690 con 0x7fc03406fa10 2024-04-26T21:08:38.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.416+0000 7fc03c000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc0340b4ea0 con 0x7fc0341f5700 2024-04-26T21:08:38.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.416+0000 7fc039600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc01c004800 0x7fc03406ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:38.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.416+0000 7fc039600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc01c004800 0x7fc03406ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42264/0 (socket says 172.21.15.26:42264) 2024-04-26T21:08:38.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.416+0000 7fc039600640 1 -- 172.21.15.26:0/2348530088 learned_addr learned my addr 172.21.15.26:0/2348530088 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:38.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.417+0000 7fc038c00640 1 --2- 172.21.15.26:0/2348530088 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0341f5700 0x7fc0340b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:38.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.417+0000 7fc039600640 1 -- 172.21.15.26:0/2348530088 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0341f5700 msgr2=0x7fc0340b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:38.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.417+0000 7fc039600640 1 --2- 172.21.15.26:0/2348530088 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0341f5700 0x7fc0340b2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.417+0000 7fc039600640 1 -- 172.21.15.26:0/2348530088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc03406fa10 msgr2=0x7fc03406fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:38.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.417+0000 7fc039600640 1 --2- 172.21.15.26:0/2348530088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc03406fa10 0x7fc03406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.417+0000 7fc039600640 1 -- 172.21.15.26:0/2348530088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0340b2550 con 0x7fc01c004800 2024-04-26T21:08:38.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.417+0000 7fc039600640 1 --2- 172.21.15.26:0/2348530088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc01c004800 0x7fc03406ef30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc024011630 tx=0x7fc024011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:38.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.417+0000 7fc031600640 1 -- 172.21.15.26:0/2348530088 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02400f920 con 0x7fc01c004800 2024-04-26T21:08:38.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.417+0000 7fc031600640 1 -- 172.21.15.26:0/2348530088 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02400fac0 con 0x7fc01c004800 2024-04-26T21:08:38.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.417+0000 7fc031600640 1 -- 172.21.15.26:0/2348530088 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02400fe70 con 0x7fc01c004800 2024-04-26T21:08:38.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.417+0000 7fc03c000640 1 -- 172.21.15.26:0/2348530088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0341f5fc0 con 0x7fc01c004800 2024-04-26T21:08:38.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.418+0000 7fc03c000640 1 -- 172.21.15.26:0/2348530088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0341f6580 con 0x7fc01c004800 2024-04-26T21:08:38.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.419+0000 7fc03c000640 1 -- 172.21.15.26:0/2348530088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc03406f550 con 0x7fc01c004800 2024-04-26T21:08:38.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.420+0000 7fc031600640 1 -- 172.21.15.26:0/2348530088 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc024015460 con 0x7fc01c004800 2024-04-26T21:08:38.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.421+0000 7fc031600640 1 --2- 172.21.15.26:0/2348530088 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc01806c8f0 0x7fc01806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:38.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.421+0000 7fc031600640 1 -- 172.21.15.26:0/2348530088 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc02408c0e0 con 0x7fc01c004800 2024-04-26T21:08:38.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.421+0000 7fc038c00640 1 --2- 172.21.15.26:0/2348530088 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc01806c8f0 0x7fc01806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:38.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.422+0000 7fc038c00640 1 --2- 172.21.15.26:0/2348530088 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc01806c8f0 0x7fc01806edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc028032000 tx=0x7fc02802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:38.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.426+0000 7fc031600640 1 -- 172.21.15.26:0/2348530088 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc03406f550 con 0x7fc01c004800 2024-04-26T21:08:38.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.628+0000 7fc03c000640 1 -- 172.21.15.26:0/2348530088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc03406a700 con 0x7fc01c004800 2024-04-26T21:08:38.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.632+0000 7fc031600640 1 -- 172.21.15.26:0/2348530088 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16570 (secure 0 0 0) 0x7fc024019030 con 0x7fc01c004800 2024-04-26T21:08:38.633 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:38.633 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:08:37.622946+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:08:35.481169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:08:36.609498+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:08:38.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.634+0000 7fc03c000640 1 -- 172.21.15.26:0/2348530088 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc01806c8f0 msgr2=0x7fc01806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:38.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.634+0000 7fc03c000640 1 --2- 172.21.15.26:0/2348530088 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc01806c8f0 0x7fc01806edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc028032000 tx=0x7fc02802e000 comp rx=0 tx=0).stop 2024-04-26T21:08:38.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.634+0000 7fc03c000640 1 -- 172.21.15.26:0/2348530088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc01c004800 msgr2=0x7fc03406ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:38.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.634+0000 7fc03c000640 1 --2- 172.21.15.26:0/2348530088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc01c004800 0x7fc03406ef30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc024011630 tx=0x7fc024011af0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.634+0000 7fc03c000640 1 -- 172.21.15.26:0/2348530088 shutdown_connections 2024-04-26T21:08:38.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.634+0000 7fc03c000640 1 --2- 172.21.15.26:0/2348530088 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc01806c8f0 0x7fc01806edb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.634+0000 7fc03c000640 1 --2- 172.21.15.26:0/2348530088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc03406fa10 0x7fc03406fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.634+0000 7fc03c000640 1 --2- 172.21.15.26:0/2348530088 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0341f5700 0x7fc0340b2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.634+0000 7fc03c000640 1 --2- 172.21.15.26:0/2348530088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc01c004800 0x7fc03406ef30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.634+0000 7fc03c000640 1 -- 172.21.15.26:0/2348530088 >> 172.21.15.26:0/2348530088 conn(0x7fc0340d1620 msgr2=0x7fc0340b6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:38.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.635+0000 7fc03c000640 1 -- 172.21.15.26:0/2348530088 shutdown_connections 2024-04-26T21:08:38.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.635+0000 7fc03c000640 1 -- 172.21.15.26:0/2348530088 wait complete. 2024-04-26T21:08:38.653 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:08:38.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.862+0000 7fcad9400640 1 Processor -- start 2024-04-26T21:08:38.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.862+0000 7fcad9400640 1 -- start start 2024-04-26T21:08:38.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.864+0000 7fcad9400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcad4145c20 0x7fcad40d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:38.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.864+0000 7fcad9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcad4146450 0x7fcad40d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:38.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.864+0000 7fcad9400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcad41468e0 0x7fcad40d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:38.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.864+0000 7fcad9400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fcad40ced10 con 0x7fcad4145c20 2024-04-26T21:08:38.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.864+0000 7fcad9400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcad40cf690 con 0x7fcad4146450 2024-04-26T21:08:38.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.864+0000 7fcad9400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcad40cf930 con 0x7fcad41468e0 2024-04-26T21:08:38.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.864+0000 7fcad3e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcad4145c20 0x7fcad40d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:38.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.864+0000 7fcad3e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcad4145c20 0x7fcad40d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34376/0 (socket says 172.21.15.26:34376) 2024-04-26T21:08:38.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.864+0000 7fcad3e00640 1 -- 172.21.15.26:0/3057627550 learned_addr learned my addr 172.21.15.26:0/3057627550 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.865+0000 7fcad3400640 1 --2- 172.21.15.26:0/3057627550 >> v2:172.21.15.86:3300/0 conn(0x7fcad4146450 0x7fcad40d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.865+0000 7fcad8a00640 1 --2- 172.21.15.26:0/3057627550 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcad41468e0 0x7fcad40d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.865+0000 7fcad3e00640 1 -- 172.21.15.26:0/3057627550 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcad41468e0 msgr2=0x7fcad40d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.865+0000 7fcad3e00640 1 --2- 172.21.15.26:0/3057627550 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcad41468e0 0x7fcad40d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.865+0000 7fcad3e00640 1 -- 172.21.15.26:0/3057627550 >> v2:172.21.15.86:3300/0 conn(0x7fcad4146450 msgr2=0x7fcad40d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.865+0000 7fcad3e00640 1 --2- 172.21.15.26:0/3057627550 >> v2:172.21.15.86:3300/0 conn(0x7fcad4146450 0x7fcad40d1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.865+0000 7fcad3e00640 1 -- 172.21.15.26:0/3057627550 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad40cfc10 con 0x7fcad4145c20 2024-04-26T21:08:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.865+0000 7fcad3400640 1 --2- 172.21.15.26:0/3057627550 >> v2:172.21.15.86:3300/0 conn(0x7fcad4146450 0x7fcad40d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.865+0000 7fcad8a00640 1 --2- 172.21.15.26:0/3057627550 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcad41468e0 0x7fcad40d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.866+0000 7fcad3e00640 1 --2- 172.21.15.26:0/3057627550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcad4145c20 0x7fcad40d06f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcad40745b0 tx=0x7fcac402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38affb5b707f3865 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.866+0000 7fcad2a00640 1 -- 172.21.15.26:0/3057627550 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac4037070 con 0x7fcad4145c20 2024-04-26T21:08:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.866+0000 7fcad2a00640 1 -- 172.21.15.26:0/3057627550 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcac4032440 con 0x7fcad4145c20 2024-04-26T21:08:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.866+0000 7fcad2a00640 1 -- 172.21.15.26:0/3057627550 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac4032810 con 0x7fcad4145c20 2024-04-26T21:08:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.867+0000 7fcad9400640 1 -- 172.21.15.26:0/3057627550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcad4145c20 msgr2=0x7fcad40d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.867+0000 7fcad9400640 1 --2- 172.21.15.26:0/3057627550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcad4145c20 0x7fcad40d06f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcad40745b0 tx=0x7fcac402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.867+0000 7fcad9400640 1 -- 172.21.15.26:0/3057627550 shutdown_connections 2024-04-26T21:08:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.867+0000 7fcad9400640 1 --2- 172.21.15.26:0/3057627550 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcad41468e0 0x7fcad40d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.867+0000 7fcad9400640 1 --2- 172.21.15.26:0/3057627550 >> v2:172.21.15.86:3300/0 conn(0x7fcad4146450 0x7fcad40d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.867+0000 7fcad9400640 1 --2- 172.21.15.26:0/3057627550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcad4145c20 0x7fcad40d06f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.867+0000 7fcad9400640 1 -- 172.21.15.26:0/3057627550 >> 172.21.15.26:0/3057627550 conn(0x7fcad406c690 msgr2=0x7fcad406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:38.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.867+0000 7fcad9400640 1 -- 172.21.15.26:0/3057627550 shutdown_connections 2024-04-26T21:08:38.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.867+0000 7fcad9400640 1 -- 172.21.15.26:0/3057627550 wait complete. 2024-04-26T21:08:38.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.869+0000 7fcad9400640 1 Processor -- start 2024-04-26T21:08:38.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.869+0000 7fcad9400640 1 -- start start 2024-04-26T21:08:38.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.870+0000 7fcad9400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcad4145c20 0x7fcad40b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:38.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.870+0000 7fcad9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcad4146450 0x7fcad4156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:38.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.870+0000 7fcad9400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcad41468e0 0x7fcad4141700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:38.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.870+0000 7fcad9400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4146000 con 0x7fcad4145c20 2024-04-26T21:08:38.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.870+0000 7fcad9400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcad40d01d0 con 0x7fcad4146450 2024-04-26T21:08:38.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.870+0000 7fcad9400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcad4157390 con 0x7fcad41468e0 2024-04-26T21:08:38.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.870+0000 7fcad3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcad4145c20 0x7fcad40b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:38.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.870+0000 7fcad3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcad4145c20 0x7fcad40b3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34388/0 (socket says 172.21.15.26:34388) 2024-04-26T21:08:38.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.870+0000 7fcad3e00640 1 -- 172.21.15.26:0/3347818409 learned_addr learned my addr 172.21.15.26:0/3347818409 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:38.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.871+0000 7fcad8a00640 1 --2- 172.21.15.26:0/3347818409 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcad41468e0 0x7fcad4141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:38.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.871+0000 7fcad3400640 1 --2- 172.21.15.26:0/3347818409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcad4146450 0x7fcad4156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:38.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.871+0000 7fcad3e00640 1 -- 172.21.15.26:0/3347818409 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcad41468e0 msgr2=0x7fcad4141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:38.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.871+0000 7fcad3e00640 1 --2- 172.21.15.26:0/3347818409 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcad41468e0 0x7fcad4141700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.871+0000 7fcad3e00640 1 -- 172.21.15.26:0/3347818409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcad4146450 msgr2=0x7fcad4156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:38.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.871+0000 7fcad3e00640 1 --2- 172.21.15.26:0/3347818409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcad4146450 0x7fcad4156c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:38.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.871+0000 7fcad3e00640 1 -- 172.21.15.26:0/3347818409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad4157670 con 0x7fcad4145c20 2024-04-26T21:08:38.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.871+0000 7fcad3e00640 1 --2- 172.21.15.26:0/3347818409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcad4145c20 0x7fcad40b3ec0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcac4000c00 tx=0x7fcac40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.871+0000 7fcacbe00640 1 -- 172.21.15.26:0/3347818409 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac4041020 con 0x7fcad4145c20 2024-04-26T21:08:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.871+0000 7fcacbe00640 1 -- 172.21.15.26:0/3347818409 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcac4030800 con 0x7fcad4145c20 2024-04-26T21:08:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.871+0000 7fcacbe00640 1 -- 172.21.15.26:0/3347818409 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac4037040 con 0x7fcad4145c20 2024-04-26T21:08:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.872+0000 7fcad9400640 1 -- 172.21.15.26:0/3347818409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad4157950 con 0x7fcad4145c20 2024-04-26T21:08:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.872+0000 7fcad9400640 1 -- 172.21.15.26:0/3347818409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad41f4b60 con 0x7fcad4145c20 2024-04-26T21:08:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.873+0000 7fcad9400640 1 -- 172.21.15.26:0/3347818409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad406a700 con 0x7fcad4145c20 2024-04-26T21:08:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.874+0000 7fcacbe00640 1 -- 172.21.15.26:0/3347818409 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcac4044020 con 0x7fcad4145c20 2024-04-26T21:08:38.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.875+0000 7fcacbe00640 1 --2- 172.21.15.26:0/3347818409 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcab406c8f0 0x7fcab406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:38.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.875+0000 7fcacbe00640 1 -- 172.21.15.26:0/3347818409 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcac40ac440 con 0x7fcad4145c20 2024-04-26T21:08:38.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.875+0000 7fcad3400640 1 --2- 172.21.15.26:0/3347818409 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcab406c8f0 0x7fcab406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:38.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.876+0000 7fcad3400640 1 --2- 172.21.15.26:0/3347818409 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcab406c8f0 0x7fcab406edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcac0006fd0 tx=0x7fcac0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:38.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:38.880+0000 7fcacbe00640 1 -- 172.21.15.26:0/3347818409 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcac404b030 con 0x7fcad4145c20 2024-04-26T21:08:39.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.083+0000 7fcad9400640 1 -- 172.21.15.26:0/3347818409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcad4070150 con 0x7fcad4145c20 2024-04-26T21:08:39.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.086+0000 7fcacbe00640 1 -- 172.21.15.26:0/3347818409 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16570 (secure 0 0 0) 0x7fcac407a870 con 0x7fcad4145c20 2024-04-26T21:08:39.088 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:39.088 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:08:37.622946+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:08:35.481169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:08:36.609498+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:08:39.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.089+0000 7fcad9400640 1 -- 172.21.15.26:0/3347818409 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcab406c8f0 msgr2=0x7fcab406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:39.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.089+0000 7fcad9400640 1 --2- 172.21.15.26:0/3347818409 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcab406c8f0 0x7fcab406edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcac0006fd0 tx=0x7fcac0008040 comp rx=0 tx=0).stop 2024-04-26T21:08:39.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.089+0000 7fcad9400640 1 -- 172.21.15.26:0/3347818409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcad4145c20 msgr2=0x7fcad40b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:39.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.089+0000 7fcad9400640 1 --2- 172.21.15.26:0/3347818409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcad4145c20 0x7fcad40b3ec0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcac4000c00 tx=0x7fcac40046c0 comp rx=0 tx=0).stop 2024-04-26T21:08:39.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.089+0000 7fcad9400640 1 -- 172.21.15.26:0/3347818409 shutdown_connections 2024-04-26T21:08:39.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.089+0000 7fcad9400640 1 --2- 172.21.15.26:0/3347818409 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcab406c8f0 0x7fcab406edb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:39.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.089+0000 7fcad9400640 1 --2- 172.21.15.26:0/3347818409 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcad41468e0 0x7fcad4141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:39.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.089+0000 7fcad9400640 1 --2- 172.21.15.26:0/3347818409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcad4146450 0x7fcad4156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:39.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.089+0000 7fcad9400640 1 --2- 172.21.15.26:0/3347818409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcad4145c20 0x7fcad40b3ec0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:39.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.089+0000 7fcad9400640 1 -- 172.21.15.26:0/3347818409 >> 172.21.15.26:0/3347818409 conn(0x7fcad406c690 msgr2=0x7fcad40b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:39.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.090+0000 7fcad9400640 1 -- 172.21.15.26:0/3347818409 shutdown_connections 2024-04-26T21:08:39.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.090+0000 7fcad9400640 1 -- 172.21.15.26:0/3347818409 wait complete. 2024-04-26T21:08:39.109 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:08:39.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.320+0000 7efe95e00640 1 Processor -- start 2024-04-26T21:08:39.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.320+0000 7efe95e00640 1 -- start start 2024-04-26T21:08:39.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.321+0000 7efe95e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe9015bdc0 0x7efe900b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:39.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.322+0000 7efe95e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efe90073530 0x7efe900cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:39.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.322+0000 7efe95e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe9006fbb0 con 0x7efe9006e210 2024-04-26T21:08:39.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.322+0000 7efe95e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7efe900b4cf0 con 0x7efe90073530 2024-04-26T21:08:39.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.322+0000 7efe95e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7efe900cf280 con 0x7efe9015bdc0 2024-04-26T21:08:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.322+0000 7efe95400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efe90073530 0x7efe900cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.322+0000 7efe94a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe9015bdc0 0x7efe900b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.322+0000 7efe95400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efe90073530 0x7efe900cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.26:41522/0 (socket says 172.21.15.26:41522) 2024-04-26T21:08:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.322+0000 7efe95400640 1 -- 172.21.15.26:0/3828691481 learned_addr learned my addr 172.21.15.26:0/3828691481 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.322+0000 7efe8fe00640 1 --1- 172.21.15.26:0/3828691481 >> v1:172.21.15.86:6789/0 conn(0x7efe9006e210 0x7efe90072e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.26:33916/0 (socket says 172.21.15.26:33916) 2024-04-26T21:08:39.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.323+0000 7efe95400640 1 -- 172.21.15.26:0/3828691481 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe9015bdc0 msgr2=0x7efe900b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:39.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.323+0000 7efe95400640 1 --2- 172.21.15.26:0/3828691481 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe9015bdc0 0x7efe900b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:39.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.323+0000 7efe95400640 1 -- 172.21.15.26:0/3828691481 >> v1:172.21.15.86:6789/0 conn(0x7efe9006e210 legacy=0x7efe90072e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:39.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.323+0000 7efe95400640 1 -- 172.21.15.26:0/3828691481 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9006e780 con 0x7efe90073530 2024-04-26T21:08:39.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.324+0000 7efe95400640 1 --2- 172.21.15.26:0/3828691481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe90073530 0x7efe900cf710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efe88009e50 tx=0x7efe8802c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5fc6be3263016a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:39.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.325+0000 7efe8f400640 1 -- 172.21.15.26:0/3828691481 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe88039070 con 0x7efe90073530 2024-04-26T21:08:39.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.325+0000 7efe8f400640 1 -- 172.21.15.26:0/3828691481 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe88034680 con 0x7efe90073530 2024-04-26T21:08:39.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.325+0000 7efe8f400640 1 -- 172.21.15.26:0/3828691481 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe88034a50 con 0x7efe90073530 2024-04-26T21:08:39.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.325+0000 7efe95e00640 1 -- 172.21.15.26:0/3828691481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe90073530 msgr2=0x7efe900cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:39.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.325+0000 7efe95e00640 1 --2- 172.21.15.26:0/3828691481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe90073530 0x7efe900cf710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efe88009e50 tx=0x7efe8802c890 comp rx=0 tx=0).stop 2024-04-26T21:08:39.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.325+0000 7efe95e00640 1 -- 172.21.15.26:0/3828691481 shutdown_connections 2024-04-26T21:08:39.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.325+0000 7efe95e00640 1 --2- 172.21.15.26:0/3828691481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe90073530 0x7efe900cf710 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:39.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.325+0000 7efe95e00640 1 --2- 172.21.15.26:0/3828691481 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe9015bdc0 0x7efe900b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:39.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.325+0000 7efe95e00640 1 -- 172.21.15.26:0/3828691481 >> 172.21.15.26:0/3828691481 conn(0x7efe900b2a10 msgr2=0x7efe900b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:39.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.326+0000 7efe95e00640 1 -- 172.21.15.26:0/3828691481 shutdown_connections 2024-04-26T21:08:39.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.326+0000 7efe95e00640 1 -- 172.21.15.26:0/3828691481 wait complete. 2024-04-26T21:08:39.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.327+0000 7efe95e00640 1 Processor -- start 2024-04-26T21:08:39.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.327+0000 7efe95e00640 1 -- start start 2024-04-26T21:08:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.328+0000 7efe95e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe9006e210 0x7efe900af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.328+0000 7efe95e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe90073530 0x7efe9014d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.328+0000 7efe95e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe9015bdc0 0x7efe90152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.328+0000 7efe95e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efe90072380 con 0x7efe9015bdc0 2024-04-26T21:08:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.328+0000 7efe95e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe90070130 con 0x7efe90073530 2024-04-26T21:08:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.328+0000 7efe95e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7efe9014e150 con 0x7efe9006e210 2024-04-26T21:08:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.329+0000 7efe95400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe9015bdc0 0x7efe90152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.329+0000 7efe95400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe9015bdc0 0x7efe90152510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34398/0 (socket says 172.21.15.26:34398) 2024-04-26T21:08:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.329+0000 7efe95400640 1 -- 172.21.15.26:0/1663710120 learned_addr learned my addr 172.21.15.26:0/1663710120 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.329+0000 7efe94a00640 1 --2- 172.21.15.26:0/1663710120 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe9006e210 0x7efe900af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:39.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.329+0000 7efe8fe00640 1 --2- 172.21.15.26:0/1663710120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe90073530 0x7efe9014d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:39.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.329+0000 7efe95400640 1 -- 172.21.15.26:0/1663710120 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe9006e210 msgr2=0x7efe900af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:39.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.329+0000 7efe95400640 1 --2- 172.21.15.26:0/1663710120 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe9006e210 0x7efe900af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:39.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.329+0000 7efe95400640 1 -- 172.21.15.26:0/1663710120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe90073530 msgr2=0x7efe9014d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:39.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.329+0000 7efe95400640 1 --2- 172.21.15.26:0/1663710120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe90073530 0x7efe9014d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:39.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.329+0000 7efe95400640 1 -- 172.21.15.26:0/1663710120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9014dc20 con 0x7efe9015bdc0 2024-04-26T21:08:39.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.330+0000 7efe95400640 1 --2- 172.21.15.26:0/1663710120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe9015bdc0 0x7efe90152510 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efe88037040 tx=0x7efe88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.330+0000 7efe8cc00640 1 -- 172.21.15.26:0/1663710120 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe88039040 con 0x7efe9015bdc0 2024-04-26T21:08:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.330+0000 7efe8cc00640 1 -- 172.21.15.26:0/1663710120 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe88042070 con 0x7efe9015bdc0 2024-04-26T21:08:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.330+0000 7efe8cc00640 1 -- 172.21.15.26:0/1663710120 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe88034e20 con 0x7efe9015bdc0 2024-04-26T21:08:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.330+0000 7efe95e00640 1 -- 172.21.15.26:0/1663710120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe900afe70 con 0x7efe9015bdc0 2024-04-26T21:08:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.330+0000 7efe95e00640 1 -- 172.21.15.26:0/1663710120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe901549a0 con 0x7efe9015bdc0 2024-04-26T21:08:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.331+0000 7efe95e00640 1 -- 172.21.15.26:0/1663710120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe90073e20 con 0x7efe9015bdc0 2024-04-26T21:08:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.332+0000 7efe8cc00640 1 -- 172.21.15.26:0/1663710120 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe88047460 con 0x7efe9015bdc0 2024-04-26T21:08:39.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.333+0000 7efe8cc00640 1 --2- 172.21.15.26:0/1663710120 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7efe7006c8f0 0x7efe7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:39.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.333+0000 7efe8cc00640 1 -- 172.21.15.26:0/1663710120 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efe880abde0 con 0x7efe9015bdc0 2024-04-26T21:08:39.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.333+0000 7efe94a00640 1 --2- 172.21.15.26:0/1663710120 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7efe7006c8f0 0x7efe7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:39.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.334+0000 7efe94a00640 1 --2- 172.21.15.26:0/1663710120 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7efe7006c8f0 0x7efe7006edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efe84009a30 tx=0x7efe84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:39.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.337+0000 7efe8cc00640 1 -- 172.21.15.26:0/1663710120 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efe90073e20 con 0x7efe9015bdc0 2024-04-26T21:08:39.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:39.576+0000 7efe95e00640 1 -- 172.21.15.26:0/1663710120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7efe9006a700 con 0x7efe9015bdc0 2024-04-26T21:08:40.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.201+0000 7efe8cc00640 1 -- 172.21.15.26:0/1663710120 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7efe8807a200 con 0x7efe9015bdc0 2024-04-26T21:08:40.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.205+0000 7efe95e00640 1 -- 172.21.15.26:0/1663710120 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7efe7006c8f0 msgr2=0x7efe7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:40.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.205+0000 7efe95e00640 1 --2- 172.21.15.26:0/1663710120 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7efe7006c8f0 0x7efe7006edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efe84009a30 tx=0x7efe84008040 comp rx=0 tx=0).stop 2024-04-26T21:08:40.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.205+0000 7efe95e00640 1 -- 172.21.15.26:0/1663710120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe9015bdc0 msgr2=0x7efe90152510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:40.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.205+0000 7efe95e00640 1 --2- 172.21.15.26:0/1663710120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe9015bdc0 0x7efe90152510 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efe88037040 tx=0x7efe88004a40 comp rx=0 tx=0).stop 2024-04-26T21:08:40.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.206+0000 7efe95e00640 1 -- 172.21.15.26:0/1663710120 shutdown_connections 2024-04-26T21:08:40.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.206+0000 7efe95e00640 1 --2- 172.21.15.26:0/1663710120 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7efe7006c8f0 0x7efe7006edb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:40.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.206+0000 7efe95e00640 1 --2- 172.21.15.26:0/1663710120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe9015bdc0 0x7efe90152510 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:40.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.206+0000 7efe95e00640 1 --2- 172.21.15.26:0/1663710120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe90073530 0x7efe9014d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:40.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.206+0000 7efe95e00640 1 --2- 172.21.15.26:0/1663710120 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe9006e210 0x7efe900af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:40.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.206+0000 7efe95e00640 1 -- 172.21.15.26:0/1663710120 >> 172.21.15.26:0/1663710120 conn(0x7efe900b2a10 msgr2=0x7efe90072e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:40.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.206+0000 7efe95e00640 1 -- 172.21.15.26:0/1663710120 shutdown_connections 2024-04-26T21:08:40.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.206+0000 7efe95e00640 1 -- 172.21.15.26:0/1663710120 wait complete. 2024-04-26T21:08:40.230 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T21:08:40.230 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:08:40.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.447+0000 7f6807400640 1 Processor -- start 2024-04-26T21:08:40.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.447+0000 7f6807400640 1 -- start start 2024-04-26T21:08:40.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.449+0000 7f6807400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68000d39f0 0x7f68000d5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:40.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.449+0000 7f6807400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f680015f710 0x7f68000d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:40.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.449+0000 7f6807400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68000d0ef0 con 0x7f68000d3dd0 2024-04-26T21:08:40.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.449+0000 7f6807400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f68000b4900 con 0x7f680015f710 2024-04-26T21:08:40.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.449+0000 7f6807400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f68000b4be0 con 0x7f68000d39f0 2024-04-26T21:08:40.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.449+0000 7f6805400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f680015f710 0x7f68000d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:40.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.449+0000 7f6805400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f680015f710 0x7f68000d07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34412/0 (socket says 172.21.15.26:34412) 2024-04-26T21:08:40.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.449+0000 7f6805400640 1 -- 172.21.15.26:0/2612991994 learned_addr learned my addr 172.21.15.26:0/2612991994 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:40.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.449+0000 7f6804a00640 1 --2- 172.21.15.26:0/2612991994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68000d39f0 0x7f68000d5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:40.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.450+0000 7f6805400640 1 -- 172.21.15.26:0/2612991994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68000d39f0 msgr2=0x7f68000d5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:40.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.450+0000 7f6805400640 1 --2- 172.21.15.26:0/2612991994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68000d39f0 0x7f68000d5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:40.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.450+0000 7f6805400640 1 -- 172.21.15.26:0/2612991994 >> v1:172.21.15.86:6789/0 conn(0x7f68000d3dd0 legacy=0x7f680015f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:40.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.450+0000 7f6805400640 1 -- 172.21.15.26:0/2612991994 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68000b4ec0 con 0x7f680015f710 2024-04-26T21:08:40.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.451+0000 7f6805400640 1 --2- 172.21.15.26:0/2612991994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f680015f710 0x7f68000d07c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009e50 tx=0x7f67f802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc143a46b7e2cf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:40.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.452+0000 7f67ff400640 1 -- 172.21.15.26:0/2612991994 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f8036070 con 0x7f680015f710 2024-04-26T21:08:40.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.452+0000 7f67ff400640 1 -- 172.21.15.26:0/2612991994 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67f8031600 con 0x7f680015f710 2024-04-26T21:08:40.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.452+0000 7f67ff400640 1 -- 172.21.15.26:0/2612991994 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f80319d0 con 0x7f680015f710 2024-04-26T21:08:40.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.452+0000 7f6807400640 1 -- 172.21.15.26:0/2612991994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f680015f710 msgr2=0x7f68000d07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:40.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.452+0000 7f6807400640 1 --2- 172.21.15.26:0/2612991994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f680015f710 0x7f68000d07c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009e50 tx=0x7f67f802a710 comp rx=0 tx=0).stop 2024-04-26T21:08:40.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.453+0000 7f6807400640 1 -- 172.21.15.26:0/2612991994 shutdown_connections 2024-04-26T21:08:40.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.453+0000 7f6807400640 1 --2- 172.21.15.26:0/2612991994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f680015f710 0x7f68000d07c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:40.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.453+0000 7f6807400640 1 --2- 172.21.15.26:0/2612991994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68000d39f0 0x7f68000d5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:40.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.453+0000 7f6807400640 1 -- 172.21.15.26:0/2612991994 >> 172.21.15.26:0/2612991994 conn(0x7f68000d51a0 msgr2=0x7f68000d55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:40.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.453+0000 7f6807400640 1 -- 172.21.15.26:0/2612991994 shutdown_connections 2024-04-26T21:08:40.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.453+0000 7f6807400640 1 -- 172.21.15.26:0/2612991994 wait complete. 2024-04-26T21:08:40.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.455+0000 7f6807400640 1 Processor -- start 2024-04-26T21:08:40.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.455+0000 7f6807400640 1 -- start start 2024-04-26T21:08:40.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.456+0000 7f6807400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68000d39f0 0x7f680006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:40.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.456+0000 7f6807400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68000d3dd0 0x7f68000c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:40.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.457+0000 7f6807400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f680015f710 0x7f68000ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:40.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.457+0000 7f6807400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f68000b57b0 con 0x7f680015f710 2024-04-26T21:08:40.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.457+0000 7f6807400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68000b5630 con 0x7f68000d39f0 2024-04-26T21:08:40.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.457+0000 7f6807400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f68000b5930 con 0x7f68000d3dd0 2024-04-26T21:08:40.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.457+0000 7f6805400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f680015f710 0x7f68000ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:40.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.457+0000 7f6805400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f680015f710 0x7f68000ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34416/0 (socket says 172.21.15.26:34416) 2024-04-26T21:08:40.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.457+0000 7f6805400640 1 -- 172.21.15.26:0/446147861 learned_addr learned my addr 172.21.15.26:0/446147861 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:40.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.457+0000 7f67ffe00640 1 --2- 172.21.15.26:0/446147861 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68000d3dd0 0x7f68000c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:40.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.457+0000 7f6804a00640 1 --2- 172.21.15.26:0/446147861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68000d39f0 0x7f680006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:40.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.457+0000 7f6805400640 1 -- 172.21.15.26:0/446147861 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68000d3dd0 msgr2=0x7f68000c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:40.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.457+0000 7f6805400640 1 --2- 172.21.15.26:0/446147861 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68000d3dd0 0x7f68000c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:40.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.457+0000 7f6805400640 1 -- 172.21.15.26:0/446147861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68000d39f0 msgr2=0x7f680006d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:40.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.457+0000 7f6805400640 1 --2- 172.21.15.26:0/446147861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68000d39f0 0x7f680006d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:40.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.457+0000 7f6805400640 1 -- 172.21.15.26:0/446147861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68000cab90 con 0x7f680015f710 2024-04-26T21:08:40.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.458+0000 7f6805400640 1 --2- 172.21.15.26:0/446147861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f680015f710 0x7f68000ca4f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f67f8034040 tx=0x7f67f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:40.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.458+0000 7f67fcc00640 1 -- 172.21.15.26:0/446147861 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f8036050 con 0x7f680015f710 2024-04-26T21:08:40.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.458+0000 7f67fcc00640 1 -- 172.21.15.26:0/446147861 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67f8042070 con 0x7f680015f710 2024-04-26T21:08:40.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.458+0000 7f67fcc00640 1 -- 172.21.15.26:0/446147861 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f8009510 con 0x7f680015f710 2024-04-26T21:08:40.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.458+0000 7f6807400640 1 -- 172.21.15.26:0/446147861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68000cae70 con 0x7f680015f710 2024-04-26T21:08:40.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.458+0000 7f6807400640 1 -- 172.21.15.26:0/446147861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68000cb430 con 0x7f680015f710 2024-04-26T21:08:40.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.460+0000 7f6807400640 1 -- 172.21.15.26:0/446147861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f680006a700 con 0x7f680015f710 2024-04-26T21:08:40.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.461+0000 7f67fcc00640 1 -- 172.21.15.26:0/446147861 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f67f8007bd0 con 0x7f680015f710 2024-04-26T21:08:40.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.462+0000 7f67fcc00640 1 --2- 172.21.15.26:0/446147861 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f67e006c8f0 0x7f67e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:40.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.462+0000 7f67fcc00640 1 -- 172.21.15.26:0/446147861 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f67f80acb30 con 0x7f680015f710 2024-04-26T21:08:40.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.462+0000 7f6804a00640 1 --2- 172.21.15.26:0/446147861 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f67e006c8f0 0x7f67e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:40.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.463+0000 7f6804a00640 1 --2- 172.21.15.26:0/446147861 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f67e006c8f0 0x7f67e006edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f67f0006fd0 tx=0x7f67f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:40.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.466+0000 7f67fcc00640 1 -- 172.21.15.26:0/446147861 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f67f807ae40 con 0x7f680015f710 2024-04-26T21:08:40.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:40.713+0000 7f6807400640 1 -- 172.21.15.26:0/446147861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f68000d4f80 con 0x7f680015f710 2024-04-26T21:08:41.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.204+0000 7f67fcc00640 1 -- 172.21.15.26:0/446147861 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f67f807a740 con 0x7f680015f710 2024-04-26T21:08:41.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.208+0000 7f6807400640 1 -- 172.21.15.26:0/446147861 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f67e006c8f0 msgr2=0x7f67e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:41.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.208+0000 7f6807400640 1 --2- 172.21.15.26:0/446147861 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f67e006c8f0 0x7f67e006edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f67f0006fd0 tx=0x7f67f0008040 comp rx=0 tx=0).stop 2024-04-26T21:08:41.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.208+0000 7f6807400640 1 -- 172.21.15.26:0/446147861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f680015f710 msgr2=0x7f68000ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:41.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.208+0000 7f6807400640 1 --2- 172.21.15.26:0/446147861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f680015f710 0x7f68000ca4f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f67f8034040 tx=0x7f67f8004a40 comp rx=0 tx=0).stop 2024-04-26T21:08:41.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.208+0000 7f6807400640 1 -- 172.21.15.26:0/446147861 shutdown_connections 2024-04-26T21:08:41.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.208+0000 7f6807400640 1 --2- 172.21.15.26:0/446147861 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f67e006c8f0 0x7f67e006edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.208+0000 7f6807400640 1 --2- 172.21.15.26:0/446147861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f680015f710 0x7f68000ca4f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.208+0000 7f6807400640 1 --2- 172.21.15.26:0/446147861 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68000d3dd0 0x7f68000c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.208+0000 7f6807400640 1 --2- 172.21.15.26:0/446147861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68000d39f0 0x7f680006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.208+0000 7f6807400640 1 -- 172.21.15.26:0/446147861 >> 172.21.15.26:0/446147861 conn(0x7f68000d51a0 msgr2=0x7f68000b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:41.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.208+0000 7f6807400640 1 -- 172.21.15.26:0/446147861 shutdown_connections 2024-04-26T21:08:41.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.208+0000 7f6807400640 1 -- 172.21.15.26:0/446147861 wait complete. 2024-04-26T21:08:41.229 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:08:41.229 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:08:41.402 INFO:tasks.ceph.mds.d.smithi086.stderr:2024-04-26T21:08:41.400+0000 7f15da600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:08:41.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.437+0000 7f32e5200640 1 Processor -- start 2024-04-26T21:08:41.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.438+0000 7f32e5200640 1 -- start start 2024-04-26T21:08:41.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.439+0000 7f32e5200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f32e006f340 0x7f32e015faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:41.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.439+0000 7f32e5200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32e006f7d0 con 0x7f32e015ffe0 2024-04-26T21:08:41.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.439+0000 7f32e5200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32e006fab0 con 0x7f32e0073470 2024-04-26T21:08:41.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.439+0000 7f32e5200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f32e0073f60 con 0x7f32e006f340 2024-04-26T21:08:41.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.440+0000 7f32dfe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f32e015ffe0 0x7f32e00d11f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44694/0 (socket says 172.21.15.26:44694) 2024-04-26T21:08:41.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.440+0000 7f32dfe00640 1 -- 172.21.15.26:0/2443288558 learned_addr learned my addr 172.21.15.26:0/2443288558 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:41.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.440+0000 7f32dea00640 1 --2- 172.21.15.26:0/2443288558 >> v2:172.21.15.86:3300/0 conn(0x7f32e006f340 0x7f32e015faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:41.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.440+0000 7f32de000640 1 -- 172.21.15.26:0/2443288558 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3148423060 0 0) 0x7f32e006f7d0 con 0x7f32e015ffe0 2024-04-26T21:08:41.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.440+0000 7f32de000640 1 -- 172.21.15.26:0/2443288558 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32c4003520 con 0x7f32e015ffe0 2024-04-26T21:08:41.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.441+0000 7f32de000640 1 -- 172.21.15.26:0/2443288558 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4249166988 0 0) 0x7f32c4003520 con 0x7f32e015ffe0 2024-04-26T21:08:41.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.441+0000 7f32de000640 1 -- 172.21.15.26:0/2443288558 >> v2:172.21.15.86:3300/0 conn(0x7f32e006f340 msgr2=0x7f32e015faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:41.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.441+0000 7f32de000640 1 --2- 172.21.15.26:0/2443288558 >> v2:172.21.15.86:3300/0 conn(0x7f32e006f340 0x7f32e015faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.441+0000 7f32de000640 1 -- 172.21.15.26:0/2443288558 >> v1:172.21.15.86:6789/0 conn(0x7f32e0073470 legacy=0x7f32e0073850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:41.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.441+0000 7f32de000640 1 -- 172.21.15.26:0/2443288558 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e006fd90 con 0x7f32e015ffe0 2024-04-26T21:08:41.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.441+0000 7f32de000640 1 -- 172.21.15.26:0/2443288558 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f32d80019f0 con 0x7f32e015ffe0 2024-04-26T21:08:41.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.441+0000 7f32de000640 1 -- 172.21.15.26:0/2443288558 >> v1:172.21.15.26:6789/0 conn(0x7f32e015ffe0 legacy=0x7f32e00d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:41.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.441+0000 7f32de000640 1 --2- 172.21.15.26:0/2443288558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32c40039b0 0x7f32c4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:41.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.441+0000 7f32de000640 1 --2- 172.21.15.26:0/2443288558 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32c40043b0 0x7f32c4004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:41.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.441+0000 7f32de000640 1 --2- 172.21.15.26:0/2443288558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c4005120 0x7f32c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:41.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.441+0000 7f32de000640 1 -- 172.21.15.26:0/2443288558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f32e0073f60 con 0x7f32c40039b0 2024-04-26T21:08:41.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.441+0000 7f32de000640 1 -- 172.21.15.26:0/2443288558 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32c400c060 con 0x7f32c4005120 2024-04-26T21:08:41.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.441+0000 7f32de000640 1 -- 172.21.15.26:0/2443288558 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f32c400c260 con 0x7f32c40043b0 2024-04-26T21:08:41.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.441+0000 7f32df400640 1 --2- 172.21.15.26:0/2443288558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32c40039b0 0x7f32c4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:41.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.442+0000 7f32df400640 1 -- 172.21.15.26:0/2443288558 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32c40043b0 msgr2=0x7f32c4004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:41.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.442+0000 7f32df400640 1 --2- 172.21.15.26:0/2443288558 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32c40043b0 0x7f32c4004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.442+0000 7f32dfe00640 1 --2- 172.21.15.26:0/2443288558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c4005120 0x7f32c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:41.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.442+0000 7f32df400640 1 -- 172.21.15.26:0/2443288558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c4005120 msgr2=0x7f32c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:41.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.442+0000 7f32df400640 1 --2- 172.21.15.26:0/2443288558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32c4005120 0x7f32c400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.442+0000 7f32df400640 1 -- 172.21.15.26:0/2443288558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c4003520 con 0x7f32c40039b0 2024-04-26T21:08:41.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.442+0000 7f32dfe00640 1 -- 172.21.15.26:0/2443288558 reap_dead start 2024-04-26T21:08:41.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.443+0000 7f32df400640 1 --2- 172.21.15.26:0/2443288558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32c40039b0 0x7f32c4003d90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f32d0001910 tx=0x7f32c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9231b9a7b6a6d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:41.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.443+0000 7f32de000640 1 -- 172.21.15.26:0/2443288558 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0037070 con 0x7f32c40039b0 2024-04-26T21:08:41.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.443+0000 7f32de000640 1 -- 172.21.15.26:0/2443288558 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d002ac40 con 0x7f32c40039b0 2024-04-26T21:08:41.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.443+0000 7f32de000640 1 -- 172.21.15.26:0/2443288558 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d00014e0 con 0x7f32c40039b0 2024-04-26T21:08:41.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.443+0000 7f32e5200640 1 -- 172.21.15.26:0/2443288558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32c40039b0 msgr2=0x7f32c4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:41.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.443+0000 7f32e5200640 1 --2- 172.21.15.26:0/2443288558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32c40039b0 0x7f32c4003d90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f32d0001910 tx=0x7f32c4005830 comp rx=0 tx=0).stop 2024-04-26T21:08:41.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.444+0000 7f32e5200640 1 -- 172.21.15.26:0/2443288558 shutdown_connections 2024-04-26T21:08:41.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.444+0000 7f32e5200640 1 --2- 172.21.15.26:0/2443288558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32c40039b0 0x7f32c4003d90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.444+0000 7f32e5200640 1 -- 172.21.15.26:0/2443288558 >> 172.21.15.26:0/2443288558 conn(0x7f32e006c690 msgr2=0x7f32e006caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:41.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.444+0000 7f32e5200640 1 -- 172.21.15.26:0/2443288558 shutdown_connections 2024-04-26T21:08:41.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.444+0000 7f32e5200640 1 -- 172.21.15.26:0/2443288558 wait complete. 2024-04-26T21:08:41.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.445+0000 7f32e5200640 1 Processor -- start 2024-04-26T21:08:41.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.446+0000 7f32e5200640 1 -- start start 2024-04-26T21:08:41.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.447+0000 7f32e5200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32c40039b0 0x7f32e01465b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:41.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.447+0000 7f32e5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32e015b090 0x7f32e015b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:41.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.447+0000 7f32e5200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32e00b22f0 0x7f32e00b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:41.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.447+0000 7f32e5200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f32e00b4d40 con 0x7f32e00b22f0 2024-04-26T21:08:41.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.447+0000 7f32e5200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32e015c4b0 con 0x7f32e015b090 2024-04-26T21:08:41.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.447+0000 7f32e5200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f32e00b4ec0 con 0x7f32c40039b0 2024-04-26T21:08:41.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.447+0000 7f32dfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32e00b22f0 0x7f32e00b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:41.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.447+0000 7f32dfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32e00b22f0 0x7f32e00b2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34426/0 (socket says 172.21.15.26:34426) 2024-04-26T21:08:41.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.447+0000 7f32dfe00640 1 -- 172.21.15.26:0/89999030 learned_addr learned my addr 172.21.15.26:0/89999030 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:41.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.447+0000 7f32df400640 1 --2- 172.21.15.26:0/89999030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32c40039b0 0x7f32e01465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:41.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.447+0000 7f32dfe00640 1 -- 172.21.15.26:0/89999030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32c40039b0 msgr2=0x7f32e01465b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:41.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.447+0000 7f32dea00640 1 --2- 172.21.15.26:0/89999030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32e015b090 0x7f32e015b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:41.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.447+0000 7f32dfe00640 1 --2- 172.21.15.26:0/89999030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32c40039b0 0x7f32e01465b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.447+0000 7f32dfe00640 1 -- 172.21.15.26:0/89999030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32e015b090 msgr2=0x7f32e015b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:41.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.447+0000 7f32dfe00640 1 --2- 172.21.15.26:0/89999030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32e015b090 0x7f32e015b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.447+0000 7f32dfe00640 1 -- 172.21.15.26:0/89999030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e0146af0 con 0x7f32e00b22f0 2024-04-26T21:08:41.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.448+0000 7f32dfe00640 1 --2- 172.21.15.26:0/89999030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32e00b22f0 0x7f32e00b2770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f32e01535a0 tx=0x7f32d800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:41.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.448+0000 7f32d7400640 1 -- 172.21.15.26:0/89999030 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d8004d60 con 0x7f32e00b22f0 2024-04-26T21:08:41.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.448+0000 7f32d7400640 1 -- 172.21.15.26:0/89999030 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d800bb20 con 0x7f32e00b22f0 2024-04-26T21:08:41.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.448+0000 7f32d7400640 1 -- 172.21.15.26:0/89999030 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d8007070 con 0x7f32e00b22f0 2024-04-26T21:08:41.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.448+0000 7f32e5200640 1 -- 172.21.15.26:0/89999030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e00b3070 con 0x7f32e00b22f0 2024-04-26T21:08:41.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.448+0000 7f32e5200640 1 -- 172.21.15.26:0/89999030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e01f8c80 con 0x7f32e00b22f0 2024-04-26T21:08:41.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.449+0000 7f32e5200640 1 -- 172.21.15.26:0/89999030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e006a700 con 0x7f32e00b22f0 2024-04-26T21:08:41.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.450+0000 7f32d7400640 1 -- 172.21.15.26:0/89999030 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f32d801b060 con 0x7f32e00b22f0 2024-04-26T21:08:41.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.451+0000 7f32d7400640 1 --2- 172.21.15.26:0/89999030 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f32c006c8f0 0x7f32c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:41.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.451+0000 7f32d7400640 1 -- 172.21.15.26:0/89999030 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f32d808ba30 con 0x7f32e00b22f0 2024-04-26T21:08:41.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.451+0000 7f32df400640 1 --2- 172.21.15.26:0/89999030 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f32c006c8f0 0x7f32c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:41.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.452+0000 7f32df400640 1 --2- 172.21.15.26:0/89999030 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f32c006c8f0 0x7f32c006edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f32d0034000 tx=0x7f32d002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:41.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.456+0000 7f32d7400640 1 -- 172.21.15.26:0/89999030 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f32d8059dc0 con 0x7f32e00b22f0 2024-04-26T21:08:41.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.700+0000 7f32e5200640 1 -- 172.21.15.26:0/89999030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32e0072fe0 con 0x7f32e00b22f0 2024-04-26T21:08:41.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.701+0000 7f32d7400640 1 -- 172.21.15.26:0/89999030 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7f32d80596c0 con 0x7f32e00b22f0 2024-04-26T21:08:41.702 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:41.702 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/2695371801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2695371801},{"type":"v1","addr":"172.21.15.26:6836","nonce":2695371801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3349732696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3349732696},{"type":"v1","addr":"172.21.15.26:6837","nonce":3349732696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/2288633870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2288633870},{"type":"v1","addr":"172.21.15.86:6833","nonce":2288633870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:08:37.622844+0000","modified":"2024-04-26T21:08:40.715140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/2125360615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2125360615},{"type":"v1","addr":"172.21.15.86:6835","nonce":2125360615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4261,"qdb_cluster":[4261]},"id":1}]} 2024-04-26T21:08:41.702 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2024-04-26T21:08:41.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.704+0000 7f32e5200640 1 -- 172.21.15.26:0/89999030 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f32c006c8f0 msgr2=0x7f32c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:41.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.704+0000 7f32e5200640 1 --2- 172.21.15.26:0/89999030 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f32c006c8f0 0x7f32c006edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f32d0034000 tx=0x7f32d002e000 comp rx=0 tx=0).stop 2024-04-26T21:08:41.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.704+0000 7f32e5200640 1 -- 172.21.15.26:0/89999030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32e00b22f0 msgr2=0x7f32e00b2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:41.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.704+0000 7f32e5200640 1 --2- 172.21.15.26:0/89999030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32e00b22f0 0x7f32e00b2770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f32e01535a0 tx=0x7f32d800b4a0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.704+0000 7f32e5200640 1 -- 172.21.15.26:0/89999030 shutdown_connections 2024-04-26T21:08:41.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.704+0000 7f32e5200640 1 --2- 172.21.15.26:0/89999030 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f32c006c8f0 0x7f32c006edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.704+0000 7f32e5200640 1 --2- 172.21.15.26:0/89999030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32e00b22f0 0x7f32e00b2770 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.704+0000 7f32e5200640 1 --2- 172.21.15.26:0/89999030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32e015b090 0x7f32e015b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.704+0000 7f32e5200640 1 --2- 172.21.15.26:0/89999030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32c40039b0 0x7f32e01465b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.704+0000 7f32e5200640 1 -- 172.21.15.26:0/89999030 >> 172.21.15.26:0/89999030 conn(0x7f32e006c690 msgr2=0x7f32e00cebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:41.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.704+0000 7f32e5200640 1 -- 172.21.15.26:0/89999030 shutdown_connections 2024-04-26T21:08:41.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.704+0000 7f32e5200640 1 -- 172.21.15.26:0/89999030 wait complete. 2024-04-26T21:08:41.725 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:08:41.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.929+0000 7f6b6a200640 1 Processor -- start 2024-04-26T21:08:41.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.930+0000 7f6b6a200640 1 -- start start 2024-04-26T21:08:41.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.931+0000 7f6b6a200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b64162ba0 0x7f6b640d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:41.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.931+0000 7f6b6a200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b640d2060 0x7f6b640d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:41.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.931+0000 7f6b6a200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b640b61c0 con 0x7f6b640d1c80 2024-04-26T21:08:41.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.931+0000 7f6b6a200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f6b640b5f70 con 0x7f6b640d2060 2024-04-26T21:08:41.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.931+0000 7f6b6a200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6b640d3830 con 0x7f6b64162ba0 2024-04-26T21:08:41.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.931+0000 7f6b69800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b640d2060 0x7f6b640d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:41.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.932+0000 7f6b69800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b640d2060 0x7f6b640d3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34436/0 (socket says 172.21.15.26:34436) 2024-04-26T21:08:41.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.932+0000 7f6b69800640 1 -- 172.21.15.26:0/2249439092 learned_addr learned my addr 172.21.15.26:0/2249439092 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:41.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.932+0000 7f6b63e00640 1 --2- 172.21.15.26:0/2249439092 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b64162ba0 0x7f6b640d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:41.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.932+0000 7f6b69800640 1 -- 172.21.15.26:0/2249439092 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b64162ba0 msgr2=0x7f6b640d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:41.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.932+0000 7f6b69800640 1 --2- 172.21.15.26:0/2249439092 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b64162ba0 0x7f6b640d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.932+0000 7f6b69800640 1 -- 172.21.15.26:0/2249439092 >> v1:172.21.15.86:6789/0 conn(0x7f6b640d1c80 legacy=0x7f6b64162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:41.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.932+0000 7f6b69800640 1 -- 172.21.15.26:0/2249439092 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b640b64a0 con 0x7f6b640d2060 2024-04-26T21:08:41.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.933+0000 7f6b69800640 1 --2- 172.21.15.26:0/2249439092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b640d2060 0x7f6b640d3cc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009e80 tx=0x7f6b5c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7ab003cd894d0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:41.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.933+0000 7f6b63400640 1 -- 172.21.15.26:0/2249439092 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c036070 con 0x7f6b640d2060 2024-04-26T21:08:41.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.933+0000 7f6b63400640 1 -- 172.21.15.26:0/2249439092 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b5c031600 con 0x7f6b640d2060 2024-04-26T21:08:41.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.933+0000 7f6b63400640 1 -- 172.21.15.26:0/2249439092 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c0319d0 con 0x7f6b640d2060 2024-04-26T21:08:41.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.934+0000 7f6b6a200640 1 -- 172.21.15.26:0/2249439092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b640d2060 msgr2=0x7f6b640d3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:41.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.934+0000 7f6b6a200640 1 --2- 172.21.15.26:0/2249439092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b640d2060 0x7f6b640d3cc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009e80 tx=0x7f6b5c02a660 comp rx=0 tx=0).stop 2024-04-26T21:08:41.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.934+0000 7f6b6a200640 1 -- 172.21.15.26:0/2249439092 shutdown_connections 2024-04-26T21:08:41.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.934+0000 7f6b6a200640 1 --2- 172.21.15.26:0/2249439092 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b64162ba0 0x7f6b640d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.934+0000 7f6b6a200640 1 --2- 172.21.15.26:0/2249439092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b640d2060 0x7f6b640d3cc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.934+0000 7f6b6a200640 1 -- 172.21.15.26:0/2249439092 >> 172.21.15.26:0/2249439092 conn(0x7f6b640d5020 msgr2=0x7f6b640d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:41.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.934+0000 7f6b6a200640 1 -- 172.21.15.26:0/2249439092 shutdown_connections 2024-04-26T21:08:41.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.934+0000 7f6b6a200640 1 -- 172.21.15.26:0/2249439092 wait complete. 2024-04-26T21:08:41.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.936+0000 7f6b6a200640 1 Processor -- start 2024-04-26T21:08:41.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.936+0000 7f6b6a200640 1 -- start start 2024-04-26T21:08:41.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.937+0000 7f6b6a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b640d1c80 0x7f6b641514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:41.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.937+0000 7f6b6a200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b640d2060 0x7f6b64151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:41.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.937+0000 7f6b6a200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b64162ba0 0x7f6b640b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:41.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.937+0000 7f6b6a200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6b640d5cc0 con 0x7f6b64162ba0 2024-04-26T21:08:41.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.937+0000 7f6b6a200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6b640d5b40 con 0x7f6b640d1c80 2024-04-26T21:08:41.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.937+0000 7f6b6a200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6b640c8b70 con 0x7f6b640d2060 2024-04-26T21:08:41.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.937+0000 7f6b69800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b64162ba0 0x7f6b640b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:41.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.937+0000 7f6b69800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b64162ba0 0x7f6b640b1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34444/0 (socket says 172.21.15.26:34444) 2024-04-26T21:08:41.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.937+0000 7f6b69800640 1 -- 172.21.15.26:0/623389762 learned_addr learned my addr 172.21.15.26:0/623389762 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:41.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.938+0000 7f6b69800640 1 -- 172.21.15.26:0/623389762 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b640d2060 msgr2=0x7f6b64151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:41.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.938+0000 7f6b69800640 1 --2- 172.21.15.26:0/623389762 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b640d2060 0x7f6b64151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.938+0000 7f6b69800640 1 -- 172.21.15.26:0/623389762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b640d1c80 msgr2=0x7f6b641514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:41.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.938+0000 7f6b69800640 1 --2- 172.21.15.26:0/623389762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b640d1c80 0x7f6b641514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:41.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.938+0000 7f6b69800640 1 -- 172.21.15.26:0/623389762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b640b3910 con 0x7f6b64162ba0 2024-04-26T21:08:41.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.938+0000 7f6b69800640 1 --2- 172.21.15.26:0/623389762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b64162ba0 0x7f6b640b1c30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c034040 tx=0x7f6b5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:41.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.938+0000 7f6b60c00640 1 -- 172.21.15.26:0/623389762 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c036050 con 0x7f6b64162ba0 2024-04-26T21:08:41.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.939+0000 7f6b60c00640 1 -- 172.21.15.26:0/623389762 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b5c042070 con 0x7f6b64162ba0 2024-04-26T21:08:41.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.939+0000 7f6b60c00640 1 -- 172.21.15.26:0/623389762 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c009510 con 0x7f6b64162ba0 2024-04-26T21:08:41.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.939+0000 7f6b6a200640 1 -- 172.21.15.26:0/623389762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b641f92a0 con 0x7f6b64162ba0 2024-04-26T21:08:41.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.939+0000 7f6b6a200640 1 -- 172.21.15.26:0/623389762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b640b3e90 con 0x7f6b64162ba0 2024-04-26T21:08:41.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.940+0000 7f6b6a200640 1 -- 172.21.15.26:0/623389762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b640b7980 con 0x7f6b64162ba0 2024-04-26T21:08:41.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.941+0000 7f6b60c00640 1 -- 172.21.15.26:0/623389762 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b5c007bd0 con 0x7f6b64162ba0 2024-04-26T21:08:41.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.942+0000 7f6b60c00640 1 --2- 172.21.15.26:0/623389762 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6b4406c920 0x7f6b4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:41.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.942+0000 7f6b60c00640 1 -- 172.21.15.26:0/623389762 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6b5c0ac9b0 con 0x7f6b64162ba0 2024-04-26T21:08:41.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.942+0000 7f6b68e00640 1 --2- 172.21.15.26:0/623389762 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6b4406c920 0x7f6b4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:41.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.943+0000 7f6b68e00640 1 --2- 172.21.15.26:0/623389762 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6b4406c920 0x7f6b4406ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6b58001560 tx=0x7f6b58007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:41.946+0000 7f6b60c00640 1 -- 172.21.15.26:0/623389762 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6b640b7980 con 0x7f6b64162ba0 2024-04-26T21:08:42.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.159+0000 7f6b6a200640 1 -- 172.21.15.26:0/623389762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b6406a700 con 0x7f6b64162ba0 2024-04-26T21:08:42.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.162+0000 7f6b60c00640 1 -- 172.21.15.26:0/623389762 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16557 (secure 0 0 0) 0x7f6b5c07add0 con 0x7f6b64162ba0 2024-04-26T21:08:42.163 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:42.163 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:08:39.177854+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:08:35.481169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:08:36.609498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:08:42.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.164+0000 7f6b6a200640 1 -- 172.21.15.26:0/623389762 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6b4406c920 msgr2=0x7f6b4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:42.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.164+0000 7f6b6a200640 1 --2- 172.21.15.26:0/623389762 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6b4406c920 0x7f6b4406ede0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6b58001560 tx=0x7f6b58007990 comp rx=0 tx=0).stop 2024-04-26T21:08:42.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.164+0000 7f6b6a200640 1 -- 172.21.15.26:0/623389762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b64162ba0 msgr2=0x7f6b640b1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:42.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.164+0000 7f6b6a200640 1 --2- 172.21.15.26:0/623389762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b64162ba0 0x7f6b640b1c30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c034040 tx=0x7f6b5c004a40 comp rx=0 tx=0).stop 2024-04-26T21:08:42.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.165+0000 7f6b6a200640 1 -- 172.21.15.26:0/623389762 shutdown_connections 2024-04-26T21:08:42.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.165+0000 7f6b6a200640 1 --2- 172.21.15.26:0/623389762 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6b4406c920 0x7f6b4406ede0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.165+0000 7f6b6a200640 1 --2- 172.21.15.26:0/623389762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b64162ba0 0x7f6b640b1c30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.165+0000 7f6b6a200640 1 --2- 172.21.15.26:0/623389762 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b640d2060 0x7f6b64151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.165+0000 7f6b6a200640 1 --2- 172.21.15.26:0/623389762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b640d1c80 0x7f6b641514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.165+0000 7f6b6a200640 1 -- 172.21.15.26:0/623389762 >> 172.21.15.26:0/623389762 conn(0x7f6b640d5020 msgr2=0x7f6b6415f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:42.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.165+0000 7f6b6a200640 1 -- 172.21.15.26:0/623389762 shutdown_connections 2024-04-26T21:08:42.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.165+0000 7f6b6a200640 1 -- 172.21.15.26:0/623389762 wait complete. 2024-04-26T21:08:42.186 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:08:42.186 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:08:42.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.379+0000 7fbb9ca00640 1 Processor -- start 2024-04-26T21:08:42.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.379+0000 7fbb9ca00640 1 -- start start 2024-04-26T21:08:42.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.380+0000 7fbb9ca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbb98146990 0x7fbb980b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:42.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.381+0000 7fbb9ca00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbb9806ec20 0x7fbb980d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:42.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.381+0000 7fbb9ca00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb980b4480 con 0x7fbb980d1210 2024-04-26T21:08:42.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.381+0000 7fbb9ca00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fbb98146d70 con 0x7fbb9806ec20 2024-04-26T21:08:42.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.381+0000 7fbb9ca00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbb980d1640 con 0x7fbb98146990 2024-04-26T21:08:42.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.381+0000 7fbb8d400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbb9806ec20 0x7fbb980d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:42.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.381+0000 7fbb8d400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbb9806ec20 0x7fbb980d0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34450/0 (socket says 172.21.15.26:34450) 2024-04-26T21:08:42.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.381+0000 7fbb8d400640 1 -- 172.21.15.26:0/183986710 learned_addr learned my addr 172.21.15.26:0/183986710 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:42.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.381+0000 7fbb8d400640 1 -- 172.21.15.26:0/183986710 >> v2:172.21.15.86:3300/0 conn(0x7fbb98146990 msgr2=0x7fbb980b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:42.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.381+0000 7fbb8d400640 1 --2- 172.21.15.26:0/183986710 >> v2:172.21.15.86:3300/0 conn(0x7fbb98146990 0x7fbb980b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.381+0000 7fbb8d400640 1 -- 172.21.15.26:0/183986710 >> v1:172.21.15.86:6789/0 conn(0x7fbb980d1210 legacy=0x7fbb98072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:42.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.381+0000 7fbb8d400640 1 -- 172.21.15.26:0/183986710 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb98073b70 con 0x7fbb9806ec20 2024-04-26T21:08:42.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.382+0000 7fbb8d400640 1 --2- 172.21.15.26:0/183986710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb9806ec20 0x7fbb980d0cd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbb980b70b0 tx=0x7fbb8002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c8baa579895dc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:42.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.383+0000 7fbb95400640 1 -- 172.21.15.26:0/183986710 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb80037070 con 0x7fbb9806ec20 2024-04-26T21:08:42.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.383+0000 7fbb95400640 1 -- 172.21.15.26:0/183986710 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb80032440 con 0x7fbb9806ec20 2024-04-26T21:08:42.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.383+0000 7fbb95400640 1 -- 172.21.15.26:0/183986710 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb80032810 con 0x7fbb9806ec20 2024-04-26T21:08:42.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.383+0000 7fbb9ca00640 1 -- 172.21.15.26:0/183986710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb9806ec20 msgr2=0x7fbb980d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:42.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.383+0000 7fbb9ca00640 1 --2- 172.21.15.26:0/183986710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb9806ec20 0x7fbb980d0cd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbb980b70b0 tx=0x7fbb8002a230 comp rx=0 tx=0).stop 2024-04-26T21:08:42.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.383+0000 7fbb9ca00640 1 -- 172.21.15.26:0/183986710 shutdown_connections 2024-04-26T21:08:42.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.383+0000 7fbb9ca00640 1 --2- 172.21.15.26:0/183986710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb9806ec20 0x7fbb980d0cd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.383+0000 7fbb9ca00640 1 --2- 172.21.15.26:0/183986710 >> v2:172.21.15.86:3300/0 conn(0x7fbb98146990 0x7fbb980b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.383+0000 7fbb9ca00640 1 -- 172.21.15.26:0/183986710 >> 172.21.15.26:0/183986710 conn(0x7fbb980cd840 msgr2=0x7fbb980cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:42.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.384+0000 7fbb9ca00640 1 -- 172.21.15.26:0/183986710 shutdown_connections 2024-04-26T21:08:42.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.384+0000 7fbb9ca00640 1 -- 172.21.15.26:0/183986710 wait complete. 2024-04-26T21:08:42.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.386+0000 7fbb9ca00640 1 Processor -- start 2024-04-26T21:08:42.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.386+0000 7fbb9ca00640 1 -- start start 2024-04-26T21:08:42.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.387+0000 7fbb9ca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb9806ec20 0x7fbb9815eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:42.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.387+0000 7fbb9ca00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb980d1210 0x7fbb9815c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:42.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.387+0000 7fbb9ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb98146990 0x7fbb9815c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:42.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.387+0000 7fbb9ca00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9806e420 con 0x7fbb9806ec20 2024-04-26T21:08:42.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.387+0000 7fbb9ca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb98074130 con 0x7fbb98146990 2024-04-26T21:08:42.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.387+0000 7fbb9ca00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbb9806e700 con 0x7fbb980d1210 2024-04-26T21:08:42.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.387+0000 7fbb95e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb9806ec20 0x7fbb9815eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:42.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.387+0000 7fbb95e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb9806ec20 0x7fbb9815eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34460/0 (socket says 172.21.15.26:34460) 2024-04-26T21:08:42.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.387+0000 7fbb95e00640 1 -- 172.21.15.26:0/1001339318 learned_addr learned my addr 172.21.15.26:0/1001339318 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:42.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.388+0000 7fbb96800640 1 --2- 172.21.15.26:0/1001339318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb98146990 0x7fbb9815c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:42.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.388+0000 7fbb95e00640 1 -- 172.21.15.26:0/1001339318 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb980d1210 msgr2=0x7fbb9815c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:42.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.388+0000 7fbb95e00640 1 --2- 172.21.15.26:0/1001339318 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb980d1210 0x7fbb9815c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.388+0000 7fbb95e00640 1 -- 172.21.15.26:0/1001339318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb98146990 msgr2=0x7fbb9815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:42.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.388+0000 7fbb95e00640 1 --2- 172.21.15.26:0/1001339318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb98146990 0x7fbb9815c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.388+0000 7fbb95e00640 1 -- 172.21.15.26:0/1001339318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb981615f0 con 0x7fbb9806ec20 2024-04-26T21:08:42.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.388+0000 7fbb96800640 1 --2- 172.21.15.26:0/1001339318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb98146990 0x7fbb9815c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:42.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.388+0000 7fbb95e00640 1 --2- 172.21.15.26:0/1001339318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb9806ec20 0x7fbb9815eb90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800b4d0 tx=0x7fbb8800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:42.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.388+0000 7fbb8ea00640 1 -- 172.21.15.26:0/1001339318 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88014070 con 0x7fbb9806ec20 2024-04-26T21:08:42.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.388+0000 7fbb8ea00640 1 -- 172.21.15.26:0/1001339318 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb88010870 con 0x7fbb9806ec20 2024-04-26T21:08:42.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.388+0000 7fbb8ea00640 1 -- 172.21.15.26:0/1001339318 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88010c60 con 0x7fbb9806ec20 2024-04-26T21:08:42.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.388+0000 7fbb9ca00640 1 -- 172.21.15.26:0/1001339318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb981f3290 con 0x7fbb9806ec20 2024-04-26T21:08:42.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.389+0000 7fbb9ca00640 1 -- 172.21.15.26:0/1001339318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb9815f0d0 con 0x7fbb9806ec20 2024-04-26T21:08:42.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.390+0000 7fbb9ca00640 1 -- 172.21.15.26:0/1001339318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb981f30c0 con 0x7fbb9806ec20 2024-04-26T21:08:42.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.391+0000 7fbb8ea00640 1 -- 172.21.15.26:0/1001339318 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb88017070 con 0x7fbb9806ec20 2024-04-26T21:08:42.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.391+0000 7fbb8ea00640 1 --2- 172.21.15.26:0/1001339318 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fbb7406c8f0 0x7fbb7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:42.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.392+0000 7fbb8ea00640 1 -- 172.21.15.26:0/1001339318 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbb8808b8c0 con 0x7fbb9806ec20 2024-04-26T21:08:42.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.392+0000 7fbb8d400640 1 --2- 172.21.15.26:0/1001339318 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fbb7406c8f0 0x7fbb7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:42.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.392+0000 7fbb8d400640 1 --2- 172.21.15.26:0/1001339318 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fbb7406c8f0 0x7fbb7406edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbb80034000 tx=0x7fbb8002c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:42.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.396+0000 7fbb8ea00640 1 -- 172.21.15.26:0/1001339318 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb88059d80 con 0x7fbb9806ec20 2024-04-26T21:08:42.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.594+0000 7fbb9ca00640 1 -- 172.21.15.26:0/1001339318 --> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbb9815f360 con 0x7fbb7406c8f0 2024-04-26T21:08:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.607+0000 7fbb8ea00640 1 -- 172.21.15.26:0/1001339318 <== mgr.4104 v2:172.21.15.26:6832/3130430358 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7fbb9815f360 con 0x7fbb7406c8f0 2024-04-26T21:08:42.611 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:42.613 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-26T21:08:42.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.612+0000 7fbb9ca00640 1 -- 172.21.15.26:0/1001339318 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fbb7406c8f0 msgr2=0x7fbb7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:42.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.612+0000 7fbb9ca00640 1 --2- 172.21.15.26:0/1001339318 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fbb7406c8f0 0x7fbb7406edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbb80034000 tx=0x7fbb8002c000 comp rx=0 tx=0).stop 2024-04-26T21:08:42.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.612+0000 7fbb9ca00640 1 -- 172.21.15.26:0/1001339318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb9806ec20 msgr2=0x7fbb9815eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:42.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.612+0000 7fbb9ca00640 1 --2- 172.21.15.26:0/1001339318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb9806ec20 0x7fbb9815eb90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800b4d0 tx=0x7fbb8800b990 comp rx=0 tx=0).stop 2024-04-26T21:08:42.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.612+0000 7fbb9ca00640 1 -- 172.21.15.26:0/1001339318 shutdown_connections 2024-04-26T21:08:42.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.612+0000 7fbb9ca00640 1 --2- 172.21.15.26:0/1001339318 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fbb7406c8f0 0x7fbb7406edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.612+0000 7fbb9ca00640 1 --2- 172.21.15.26:0/1001339318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb98146990 0x7fbb9815c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.612+0000 7fbb9ca00640 1 --2- 172.21.15.26:0/1001339318 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb980d1210 0x7fbb9815c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.612+0000 7fbb9ca00640 1 --2- 172.21.15.26:0/1001339318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb9806ec20 0x7fbb9815eb90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.612+0000 7fbb9ca00640 1 -- 172.21.15.26:0/1001339318 >> 172.21.15.26:0/1001339318 conn(0x7fbb980cd840 msgr2=0x7fbb980cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:42.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.612+0000 7fbb9ca00640 1 -- 172.21.15.26:0/1001339318 shutdown_connections 2024-04-26T21:08:42.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.613+0000 7fbb9ca00640 1 -- 172.21.15.26:0/1001339318 wait complete. 2024-04-26T21:08:42.631 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-26T21:08:41.286522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235764,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738956,"statfs":{"total":773094113280,"available":772852690944,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.161313"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711962+0000","last_change":"2024-04-26T21:08:38.198898+0000","last_active":"2024-04-26T21:08:39.711962+0000","last_peered":"2024-04-26T21:08:39.711962+0000","last_clean":"2024-04-26T21:08:39.711962+0000","last_became_active":"2024-04-26T21:08:38.198596+0000","last_became_peered":"2024-04-26T21:08:38.198596+0000","last_unstale":"2024-04-26T21:08:39.711962+0000","last_undegraded":"2024-04-26T21:08:39.711962+0000","last_fullsized":"2024-04-26T21:08:39.711962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:19.756896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211160+0000","last_change":"2024-04-26T21:08:38.194080+0000","last_active":"2024-04-26T21:08:39.211160+0000","last_peered":"2024-04-26T21:08:39.211160+0000","last_clean":"2024-04-26T21:08:39.211160+0000","last_became_active":"2024-04-26T21:08:38.193534+0000","last_became_peered":"2024-04-26T21:08:38.193534+0000","last_unstale":"2024-04-26T21:08:39.211160+0000","last_undegraded":"2024-04-26T21:08:39.211160+0000","last_fullsized":"2024-04-26T21:08:39.211160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:22.132136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712041+0000","last_change":"2024-04-26T21:08:38.198384+0000","last_active":"2024-04-26T21:08:39.712041+0000","last_peered":"2024-04-26T21:08:39.712041+0000","last_clean":"2024-04-26T21:08:39.712041+0000","last_became_active":"2024-04-26T21:08:38.198055+0000","last_became_peered":"2024-04-26T21:08:38.198055+0000","last_unstale":"2024-04-26T21:08:39.712041+0000","last_undegraded":"2024-04-26T21:08:39.712041+0000","last_fullsized":"2024-04-26T21:08:39.712041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:55.566474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194277+0000","last_change":"2024-04-26T21:08:38.205534+0000","last_active":"2024-04-26T21:08:39.194277+0000","last_peered":"2024-04-26T21:08:39.194277+0000","last_clean":"2024-04-26T21:08:39.194277+0000","last_became_active":"2024-04-26T21:08:38.205235+0000","last_became_peered":"2024-04-26T21:08:38.205235+0000","last_unstale":"2024-04-26T21:08:39.194277+0000","last_undegraded":"2024-04-26T21:08:39.194277+0000","last_fullsized":"2024-04-26T21:08:39.194277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:26.983477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711420+0000","last_change":"2024-04-26T21:08:38.200597+0000","last_active":"2024-04-26T21:08:39.711420+0000","last_peered":"2024-04-26T21:08:39.711420+0000","last_clean":"2024-04-26T21:08:39.711420+0000","last_became_active":"2024-04-26T21:08:38.197772+0000","last_became_peered":"2024-04-26T21:08:38.197772+0000","last_unstale":"2024-04-26T21:08:39.711420+0000","last_undegraded":"2024-04-26T21:08:39.711420+0000","last_fullsized":"2024-04-26T21:08:39.711420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:17.370683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.196006+0000","last_change":"2024-04-26T21:08:38.210251+0000","last_active":"2024-04-26T21:08:39.196006+0000","last_peered":"2024-04-26T21:08:39.196006+0000","last_clean":"2024-04-26T21:08:39.196006+0000","last_became_active":"2024-04-26T21:08:38.209628+0000","last_became_peered":"2024-04-26T21:08:38.209628+0000","last_unstale":"2024-04-26T21:08:39.196006+0000","last_undegraded":"2024-04-26T21:08:39.196006+0000","last_fullsized":"2024-04-26T21:08:39.196006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:28.069603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711557+0000","last_change":"2024-04-26T21:08:38.199522+0000","last_active":"2024-04-26T21:08:39.711557+0000","last_peered":"2024-04-26T21:08:39.711557+0000","last_clean":"2024-04-26T21:08:39.711557+0000","last_became_active":"2024-04-26T21:08:38.198122+0000","last_became_peered":"2024-04-26T21:08:38.198122+0000","last_unstale":"2024-04-26T21:08:39.711557+0000","last_undegraded":"2024-04-26T21:08:39.711557+0000","last_fullsized":"2024-04-26T21:08:39.711557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:55.797399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711087+0000","last_change":"2024-04-26T21:08:38.207966+0000","last_active":"2024-04-26T21:08:39.711087+0000","last_peered":"2024-04-26T21:08:39.711087+0000","last_clean":"2024-04-26T21:08:39.711087+0000","last_became_active":"2024-04-26T21:08:38.207617+0000","last_became_peered":"2024-04-26T21:08:38.207617+0000","last_unstale":"2024-04-26T21:08:39.711087+0000","last_undegraded":"2024-04-26T21:08:39.711087+0000","last_fullsized":"2024-04-26T21:08:39.711087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:55.566474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712128+0000","last_change":"2024-04-26T21:08:38.204121+0000","last_active":"2024-04-26T21:08:39.712128+0000","last_peered":"2024-04-26T21:08:39.712128+0000","last_clean":"2024-04-26T21:08:39.712128+0000","last_became_active":"2024-04-26T21:08:38.203125+0000","last_became_peered":"2024-04-26T21:08:38.203125+0000","last_unstale":"2024-04-26T21:08:39.712128+0000","last_undegraded":"2024-04-26T21:08:39.712128+0000","last_fullsized":"2024-04-26T21:08:39.712128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:36.343803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217352+0000","last_change":"2024-04-26T21:08:38.204308+0000","last_active":"2024-04-26T21:08:39.217352+0000","last_peered":"2024-04-26T21:08:39.217352+0000","last_clean":"2024-04-26T21:08:39.217352+0000","last_became_active":"2024-04-26T21:08:38.203942+0000","last_became_peered":"2024-04-26T21:08:38.203942+0000","last_unstale":"2024-04-26T21:08:39.217352+0000","last_undegraded":"2024-04-26T21:08:39.217352+0000","last_fullsized":"2024-04-26T21:08:39.217352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:42.506444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217214+0000","last_change":"2024-04-26T21:08:38.201929+0000","last_active":"2024-04-26T21:08:39.217214+0000","last_peered":"2024-04-26T21:08:39.217214+0000","last_clean":"2024-04-26T21:08:39.217214+0000","last_became_active":"2024-04-26T21:08:38.201646+0000","last_became_peered":"2024-04-26T21:08:38.201646+0000","last_unstale":"2024-04-26T21:08:39.217214+0000","last_undegraded":"2024-04-26T21:08:39.217214+0000","last_fullsized":"2024-04-26T21:08:39.217214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:40.618646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195574+0000","last_change":"2024-04-26T21:08:38.207847+0000","last_active":"2024-04-26T21:08:39.195574+0000","last_peered":"2024-04-26T21:08:39.195574+0000","last_clean":"2024-04-26T21:08:39.195574+0000","last_became_active":"2024-04-26T21:08:38.204908+0000","last_became_peered":"2024-04-26T21:08:38.204908+0000","last_unstale":"2024-04-26T21:08:39.195574+0000","last_undegraded":"2024-04-26T21:08:39.195574+0000","last_fullsized":"2024-04-26T21:08:39.195574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:00.992973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711183+0000","last_change":"2024-04-26T21:08:38.203775+0000","last_active":"2024-04-26T21:08:39.711183+0000","last_peered":"2024-04-26T21:08:39.711183+0000","last_clean":"2024-04-26T21:08:39.711183+0000","last_became_active":"2024-04-26T21:08:38.203139+0000","last_became_peered":"2024-04-26T21:08:38.203139+0000","last_unstale":"2024-04-26T21:08:39.711183+0000","last_undegraded":"2024-04-26T21:08:39.711183+0000","last_fullsized":"2024-04-26T21:08:39.711183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:01.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712480+0000","last_change":"2024-04-26T21:08:38.195656+0000","last_active":"2024-04-26T21:08:39.712480+0000","last_peered":"2024-04-26T21:08:39.712480+0000","last_clean":"2024-04-26T21:08:39.712480+0000","last_became_active":"2024-04-26T21:08:38.195240+0000","last_became_peered":"2024-04-26T21:08:38.195240+0000","last_unstale":"2024-04-26T21:08:39.712480+0000","last_undegraded":"2024-04-26T21:08:39.712480+0000","last_fullsized":"2024-04-26T21:08:39.712480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:16.232014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211031+0000","last_change":"2024-04-26T21:08:38.186523+0000","last_active":"2024-04-26T21:08:39.211031+0000","last_peered":"2024-04-26T21:08:39.211031+0000","last_clean":"2024-04-26T21:08:39.211031+0000","last_became_active":"2024-04-26T21:08:38.186159+0000","last_became_peered":"2024-04-26T21:08:38.186159+0000","last_unstale":"2024-04-26T21:08:39.211031+0000","last_undegraded":"2024-04-26T21:08:39.211031+0000","last_fullsized":"2024-04-26T21:08:39.211031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:40.618646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194033+0000","last_change":"2024-04-26T21:08:38.212958+0000","last_active":"2024-04-26T21:08:39.194033+0000","last_peered":"2024-04-26T21:08:39.194033+0000","last_clean":"2024-04-26T21:08:39.194033+0000","last_became_active":"2024-04-26T21:08:38.212666+0000","last_became_peered":"2024-04-26T21:08:38.212666+0000","last_unstale":"2024-04-26T21:08:39.194033+0000","last_undegraded":"2024-04-26T21:08:39.194033+0000","last_fullsized":"2024-04-26T21:08:39.194033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:42.506444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195541+0000","last_change":"2024-04-26T21:08:38.214751+0000","last_active":"2024-04-26T21:08:39.195541+0000","last_peered":"2024-04-26T21:08:39.195541+0000","last_clean":"2024-04-26T21:08:39.195541+0000","last_became_active":"2024-04-26T21:08:38.212857+0000","last_became_peered":"2024-04-26T21:08:38.212857+0000","last_unstale":"2024-04-26T21:08:39.195541+0000","last_undegraded":"2024-04-26T21:08:39.195541+0000","last_fullsized":"2024-04-26T21:08:39.195541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:35.132498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.191204+0000","last_change":"2024-04-26T21:08:38.198823+0000","last_active":"2024-04-26T21:08:39.191204+0000","last_peered":"2024-04-26T21:08:39.191204+0000","last_clean":"2024-04-26T21:08:39.191204+0000","last_became_active":"2024-04-26T21:08:38.198521+0000","last_became_peered":"2024-04-26T21:08:38.198521+0000","last_unstale":"2024-04-26T21:08:39.191204+0000","last_undegraded":"2024-04-26T21:08:39.191204+0000","last_fullsized":"2024-04-26T21:08:39.191204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:34.390026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712075+0000","last_change":"2024-04-26T21:08:38.200413+0000","last_active":"2024-04-26T21:08:39.712075+0000","last_peered":"2024-04-26T21:08:39.712075+0000","last_clean":"2024-04-26T21:08:39.712075+0000","last_became_active":"2024-04-26T21:08:38.200023+0000","last_became_peered":"2024-04-26T21:08:38.200023+0000","last_unstale":"2024-04-26T21:08:39.712075+0000","last_undegraded":"2024-04-26T21:08:39.712075+0000","last_fullsized":"2024-04-26T21:08:39.712075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:28.069603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.216976+0000","last_change":"2024-04-26T21:08:38.211354+0000","last_active":"2024-04-26T21:08:39.216976+0000","last_peered":"2024-04-26T21:08:39.216976+0000","last_clean":"2024-04-26T21:08:39.216976+0000","last_became_active":"2024-04-26T21:08:38.211016+0000","last_became_peered":"2024-04-26T21:08:38.211016+0000","last_unstale":"2024-04-26T21:08:39.216976+0000","last_undegraded":"2024-04-26T21:08:39.216976+0000","last_fullsized":"2024-04-26T21:08:39.216976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:29.659217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712867+0000","last_change":"2024-04-26T21:08:38.201685+0000","last_active":"2024-04-26T21:08:39.712867+0000","last_peered":"2024-04-26T21:08:39.712867+0000","last_clean":"2024-04-26T21:08:39.712867+0000","last_became_active":"2024-04-26T21:08:38.200467+0000","last_became_peered":"2024-04-26T21:08:38.200467+0000","last_unstale":"2024-04-26T21:08:39.712867+0000","last_undegraded":"2024-04-26T21:08:39.712867+0000","last_fullsized":"2024-04-26T21:08:39.712867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:40.618646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194361+0000","last_change":"2024-04-26T21:08:38.204033+0000","last_active":"2024-04-26T21:08:39.194361+0000","last_peered":"2024-04-26T21:08:39.194361+0000","last_clean":"2024-04-26T21:08:39.194361+0000","last_became_active":"2024-04-26T21:08:38.202425+0000","last_became_peered":"2024-04-26T21:08:38.202425+0000","last_unstale":"2024-04-26T21:08:39.194361+0000","last_undegraded":"2024-04-26T21:08:39.194361+0000","last_fullsized":"2024-04-26T21:08:39.194361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:00.305226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711233+0000","last_change":"2024-04-26T21:08:38.216188+0000","last_active":"2024-04-26T21:08:39.711233+0000","last_peered":"2024-04-26T21:08:39.711233+0000","last_clean":"2024-04-26T21:08:39.711233+0000","last_became_active":"2024-04-26T21:08:38.215866+0000","last_became_peered":"2024-04-26T21:08:38.215866+0000","last_unstale":"2024-04-26T21:08:39.711233+0000","last_undegraded":"2024-04-26T21:08:39.711233+0000","last_fullsized":"2024-04-26T21:08:39.711233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:29.659217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.193946+0000","last_change":"2024-04-26T21:08:38.213795+0000","last_active":"2024-04-26T21:08:39.193946+0000","last_peered":"2024-04-26T21:08:39.193946+0000","last_clean":"2024-04-26T21:08:39.193946+0000","last_became_active":"2024-04-26T21:08:38.211952+0000","last_became_peered":"2024-04-26T21:08:38.211952+0000","last_unstale":"2024-04-26T21:08:39.193946+0000","last_undegraded":"2024-04-26T21:08:39.193946+0000","last_fullsized":"2024-04-26T21:08:39.193946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:22.132136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712055+0000","last_change":"2024-04-26T21:08:38.211961+0000","last_active":"2024-04-26T21:08:39.712055+0000","last_peered":"2024-04-26T21:08:39.712055+0000","last_clean":"2024-04-26T21:08:39.712055+0000","last_became_active":"2024-04-26T21:08:38.211590+0000","last_became_peered":"2024-04-26T21:08:38.211590+0000","last_unstale":"2024-04-26T21:08:39.712055+0000","last_undegraded":"2024-04-26T21:08:39.712055+0000","last_fullsized":"2024-04-26T21:08:39.712055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:22.132136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217302+0000","last_change":"2024-04-26T21:08:38.200054+0000","last_active":"2024-04-26T21:08:39.217302+0000","last_peered":"2024-04-26T21:08:39.217302+0000","last_clean":"2024-04-26T21:08:39.217302+0000","last_became_active":"2024-04-26T21:08:38.199738+0000","last_became_peered":"2024-04-26T21:08:38.199738+0000","last_unstale":"2024-04-26T21:08:39.217302+0000","last_undegraded":"2024-04-26T21:08:39.217302+0000","last_fullsized":"2024-04-26T21:08:39.217302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:22.132136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.210875+0000","last_change":"2024-04-26T21:08:38.195577+0000","last_active":"2024-04-26T21:08:39.210875+0000","last_peered":"2024-04-26T21:08:39.210875+0000","last_clean":"2024-04-26T21:08:39.210875+0000","last_became_active":"2024-04-26T21:08:38.195284+0000","last_became_peered":"2024-04-26T21:08:38.195284+0000","last_unstale":"2024-04-26T21:08:39.210875+0000","last_undegraded":"2024-04-26T21:08:39.210875+0000","last_fullsized":"2024-04-26T21:08:39.210875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:29.659217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.713361+0000","last_change":"2024-04-26T21:08:38.202932+0000","last_active":"2024-04-26T21:08:39.713361+0000","last_peered":"2024-04-26T21:08:39.713361+0000","last_clean":"2024-04-26T21:08:39.713361+0000","last_became_active":"2024-04-26T21:08:38.202614+0000","last_became_peered":"2024-04-26T21:08:38.202614+0000","last_unstale":"2024-04-26T21:08:39.713361+0000","last_undegraded":"2024-04-26T21:08:39.713361+0000","last_fullsized":"2024-04-26T21:08:39.713361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:44.324217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.713283+0000","last_change":"2024-04-26T21:08:38.198429+0000","last_active":"2024-04-26T21:08:39.713283+0000","last_peered":"2024-04-26T21:08:39.713283+0000","last_clean":"2024-04-26T21:08:39.713283+0000","last_became_active":"2024-04-26T21:08:38.198144+0000","last_became_peered":"2024-04-26T21:08:38.198144+0000","last_unstale":"2024-04-26T21:08:39.713283+0000","last_undegraded":"2024-04-26T21:08:39.713283+0000","last_fullsized":"2024-04-26T21:08:39.713283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:22.132136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194610+0000","last_change":"2024-04-26T21:08:38.205890+0000","last_active":"2024-04-26T21:08:39.194610+0000","last_peered":"2024-04-26T21:08:39.194610+0000","last_clean":"2024-04-26T21:08:39.194610+0000","last_became_active":"2024-04-26T21:08:38.205583+0000","last_became_peered":"2024-04-26T21:08:38.205583+0000","last_unstale":"2024-04-26T21:08:39.194610+0000","last_undegraded":"2024-04-26T21:08:39.194610+0000","last_fullsized":"2024-04-26T21:08:39.194610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:17.370683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.190845+0000","last_change":"2024-04-26T21:08:38.204162+0000","last_active":"2024-04-26T21:08:39.190845+0000","last_peered":"2024-04-26T21:08:39.190845+0000","last_clean":"2024-04-26T21:08:39.190845+0000","last_became_active":"2024-04-26T21:08:38.203855+0000","last_became_peered":"2024-04-26T21:08:38.203855+0000","last_unstale":"2024-04-26T21:08:39.190845+0000","last_undegraded":"2024-04-26T21:08:39.190845+0000","last_fullsized":"2024-04-26T21:08:39.190845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:31.643503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.216694+0000","last_change":"2024-04-26T21:08:38.207713+0000","last_active":"2024-04-26T21:08:39.216694+0000","last_peered":"2024-04-26T21:08:39.216694+0000","last_clean":"2024-04-26T21:08:39.216694+0000","last_became_active":"2024-04-26T21:08:38.207384+0000","last_became_peered":"2024-04-26T21:08:38.207384+0000","last_unstale":"2024-04-26T21:08:39.216694+0000","last_undegraded":"2024-04-26T21:08:39.216694+0000","last_fullsized":"2024-04-26T21:08:39.216694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:55.566474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.196940+0000","last_change":"2024-04-26T21:08:38.194999+0000","last_active":"2024-04-26T21:08:39.196940+0000","last_peered":"2024-04-26T21:08:39.196940+0000","last_clean":"2024-04-26T21:08:39.196940+0000","last_became_active":"2024-04-26T21:08:38.194647+0000","last_became_peered":"2024-04-26T21:08:38.194647+0000","last_unstale":"2024-04-26T21:08:39.196940+0000","last_undegraded":"2024-04-26T21:08:39.196940+0000","last_fullsized":"2024-04-26T21:08:39.196940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:34.390026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711906+0000","last_change":"2024-04-26T21:08:37.168719+0000","last_active":"2024-04-26T21:08:39.711906+0000","last_peered":"2024-04-26T21:08:39.711906+0000","last_clean":"2024-04-26T21:08:39.711906+0000","last_became_active":"2024-04-26T21:08:37.168257+0000","last_became_peered":"2024-04-26T21:08:37.168257+0000","last_unstale":"2024-04-26T21:08:39.711906+0000","last_undegraded":"2024-04-26T21:08:39.711906+0000","last_fullsized":"2024-04-26T21:08:39.711906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:18.839848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.713404+0000","last_change":"2024-04-26T21:08:38.190358+0000","last_active":"2024-04-26T21:08:39.713404+0000","last_peered":"2024-04-26T21:08:39.713404+0000","last_clean":"2024-04-26T21:08:39.713404+0000","last_became_active":"2024-04-26T21:08:38.189905+0000","last_became_peered":"2024-04-26T21:08:38.189905+0000","last_unstale":"2024-04-26T21:08:39.713404+0000","last_undegraded":"2024-04-26T21:08:39.713404+0000","last_fullsized":"2024-04-26T21:08:39.713404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:42.506444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.216776+0000","last_change":"2024-04-26T21:08:37.177780+0000","last_active":"2024-04-26T21:08:39.216776+0000","last_peered":"2024-04-26T21:08:39.216776+0000","last_clean":"2024-04-26T21:08:39.216776+0000","last_became_active":"2024-04-26T21:08:37.177488+0000","last_became_peered":"2024-04-26T21:08:37.177488+0000","last_unstale":"2024-04-26T21:08:39.216776+0000","last_undegraded":"2024-04-26T21:08:39.216776+0000","last_fullsized":"2024-04-26T21:08:39.216776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:59.977930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711590+0000","last_change":"2024-04-26T21:08:37.171155+0000","last_active":"2024-04-26T21:08:39.711590+0000","last_peered":"2024-04-26T21:08:39.711590+0000","last_clean":"2024-04-26T21:08:39.711590+0000","last_became_active":"2024-04-26T21:08:37.170816+0000","last_became_peered":"2024-04-26T21:08:37.170816+0000","last_unstale":"2024-04-26T21:08:39.711590+0000","last_undegraded":"2024-04-26T21:08:39.711590+0000","last_fullsized":"2024-04-26T21:08:39.711590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:17.499100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712682+0000","last_change":"2024-04-26T21:08:38.206720+0000","last_active":"2024-04-26T21:08:39.712682+0000","last_peered":"2024-04-26T21:08:39.712682+0000","last_clean":"2024-04-26T21:08:39.712682+0000","last_became_active":"2024-04-26T21:08:38.206401+0000","last_became_peered":"2024-04-26T21:08:38.206401+0000","last_unstale":"2024-04-26T21:08:39.712682+0000","last_undegraded":"2024-04-26T21:08:39.712682+0000","last_fullsized":"2024-04-26T21:08:39.712682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:00.305226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194170+0000","last_change":"2024-04-26T21:08:37.161426+0000","last_active":"2024-04-26T21:08:39.194170+0000","last_peered":"2024-04-26T21:08:39.194170+0000","last_clean":"2024-04-26T21:08:39.194170+0000","last_became_active":"2024-04-26T21:08:37.161119+0000","last_became_peered":"2024-04-26T21:08:37.161119+0000","last_unstale":"2024-04-26T21:08:39.194170+0000","last_undegraded":"2024-04-26T21:08:39.194170+0000","last_fullsized":"2024-04-26T21:08:39.194170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:34.683220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211320+0000","last_change":"2024-04-26T21:08:38.195331+0000","last_active":"2024-04-26T21:08:39.211320+0000","last_peered":"2024-04-26T21:08:39.211320+0000","last_clean":"2024-04-26T21:08:39.211320+0000","last_became_active":"2024-04-26T21:08:38.195002+0000","last_became_peered":"2024-04-26T21:08:38.195002+0000","last_unstale":"2024-04-26T21:08:39.211320+0000","last_undegraded":"2024-04-26T21:08:39.211320+0000","last_fullsized":"2024-04-26T21:08:39.211320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:42.506444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217273+0000","last_change":"2024-04-26T21:08:37.172072+0000","last_active":"2024-04-26T21:08:39.217273+0000","last_peered":"2024-04-26T21:08:39.217273+0000","last_clean":"2024-04-26T21:08:39.217273+0000","last_became_active":"2024-04-26T21:08:37.171778+0000","last_became_peered":"2024-04-26T21:08:37.171778+0000","last_unstale":"2024-04-26T21:08:39.217273+0000","last_undegraded":"2024-04-26T21:08:39.217273+0000","last_fullsized":"2024-04-26T21:08:39.217273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:00.488332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194798+0000","last_change":"2024-04-26T21:08:38.202960+0000","last_active":"2024-04-26T21:08:39.194798+0000","last_peered":"2024-04-26T21:08:39.194798+0000","last_clean":"2024-04-26T21:08:39.194798+0000","last_became_active":"2024-04-26T21:08:38.202642+0000","last_became_peered":"2024-04-26T21:08:38.202642+0000","last_unstale":"2024-04-26T21:08:39.194798+0000","last_undegraded":"2024-04-26T21:08:39.194798+0000","last_fullsized":"2024-04-26T21:08:39.194798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:55.797399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194905+0000","last_change":"2024-04-26T21:08:37.174576+0000","last_active":"2024-04-26T21:08:39.194905+0000","last_peered":"2024-04-26T21:08:39.194905+0000","last_clean":"2024-04-26T21:08:39.194905+0000","last_became_active":"2024-04-26T21:08:37.174223+0000","last_became_peered":"2024-04-26T21:08:37.174223+0000","last_unstale":"2024-04-26T21:08:39.194905+0000","last_undegraded":"2024-04-26T21:08:39.194905+0000","last_fullsized":"2024-04-26T21:08:39.194905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:00.488332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712374+0000","last_change":"2024-04-26T21:08:38.210232+0000","last_active":"2024-04-26T21:08:39.712374+0000","last_peered":"2024-04-26T21:08:39.712374+0000","last_clean":"2024-04-26T21:08:39.712374+0000","last_became_active":"2024-04-26T21:08:38.207957+0000","last_became_peered":"2024-04-26T21:08:38.207957+0000","last_unstale":"2024-04-26T21:08:39.712374+0000","last_undegraded":"2024-04-26T21:08:39.712374+0000","last_fullsized":"2024-04-26T21:08:39.712374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:29.659217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211019+0000","last_change":"2024-04-26T21:08:37.166766+0000","last_active":"2024-04-26T21:08:39.211019+0000","last_peered":"2024-04-26T21:08:39.211019+0000","last_clean":"2024-04-26T21:08:39.211019+0000","last_became_active":"2024-04-26T21:08:37.166441+0000","last_became_peered":"2024-04-26T21:08:37.166441+0000","last_unstale":"2024-04-26T21:08:39.211019+0000","last_undegraded":"2024-04-26T21:08:39.211019+0000","last_fullsized":"2024-04-26T21:08:39.211019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:58.240486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195784+0000","last_change":"2024-04-26T21:08:38.209109+0000","last_active":"2024-04-26T21:08:39.195784+0000","last_peered":"2024-04-26T21:08:39.195784+0000","last_clean":"2024-04-26T21:08:39.195784+0000","last_became_active":"2024-04-26T21:08:38.208779+0000","last_became_peered":"2024-04-26T21:08:38.208779+0000","last_unstale":"2024-04-26T21:08:39.195784+0000","last_undegraded":"2024-04-26T21:08:39.195784+0000","last_fullsized":"2024-04-26T21:08:39.195784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:59.255528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217053+0000","last_change":"2024-04-26T21:08:37.176062+0000","last_active":"2024-04-26T21:08:39.217053+0000","last_peered":"2024-04-26T21:08:39.217053+0000","last_clean":"2024-04-26T21:08:39.217053+0000","last_became_active":"2024-04-26T21:08:37.175757+0000","last_became_peered":"2024-04-26T21:08:37.175757+0000","last_unstale":"2024-04-26T21:08:39.217053+0000","last_undegraded":"2024-04-26T21:08:39.217053+0000","last_fullsized":"2024-04-26T21:08:39.217053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:58.240486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711341+0000","last_change":"2024-04-26T21:08:38.199662+0000","last_active":"2024-04-26T21:08:39.711341+0000","last_peered":"2024-04-26T21:08:39.711341+0000","last_clean":"2024-04-26T21:08:39.711341+0000","last_became_active":"2024-04-26T21:08:38.198909+0000","last_became_peered":"2024-04-26T21:08:38.198909+0000","last_unstale":"2024-04-26T21:08:39.711341+0000","last_undegraded":"2024-04-26T21:08:39.711341+0000","last_fullsized":"2024-04-26T21:08:39.711341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:46.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.713546+0000","last_change":"2024-04-26T21:08:37.149949+0000","last_active":"2024-04-26T21:08:39.713546+0000","last_peered":"2024-04-26T21:08:39.713546+0000","last_clean":"2024-04-26T21:08:39.713546+0000","last_became_active":"2024-04-26T21:08:37.149579+0000","last_became_peered":"2024-04-26T21:08:37.149579+0000","last_unstale":"2024-04-26T21:08:39.713546+0000","last_undegraded":"2024-04-26T21:08:39.713546+0000","last_fullsized":"2024-04-26T21:08:39.713546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:29.045350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712396+0000","last_change":"2024-04-26T21:08:38.196010+0000","last_active":"2024-04-26T21:08:39.712396+0000","last_peered":"2024-04-26T21:08:39.712396+0000","last_clean":"2024-04-26T21:08:39.712396+0000","last_became_active":"2024-04-26T21:08:38.195650+0000","last_became_peered":"2024-04-26T21:08:38.195650+0000","last_unstale":"2024-04-26T21:08:39.712396+0000","last_undegraded":"2024-04-26T21:08:39.712396+0000","last_fullsized":"2024-04-26T21:08:39.712396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:59.807950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194744+0000","last_change":"2024-04-26T21:08:37.170877+0000","last_active":"2024-04-26T21:08:39.194744+0000","last_peered":"2024-04-26T21:08:39.194744+0000","last_clean":"2024-04-26T21:08:39.194744+0000","last_became_active":"2024-04-26T21:08:37.170548+0000","last_became_peered":"2024-04-26T21:08:37.170548+0000","last_unstale":"2024-04-26T21:08:39.194744+0000","last_undegraded":"2024-04-26T21:08:39.194744+0000","last_fullsized":"2024-04-26T21:08:39.194744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:58.240486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711189+0000","last_change":"2024-04-26T21:08:38.217551+0000","last_active":"2024-04-26T21:08:39.711189+0000","last_peered":"2024-04-26T21:08:39.711189+0000","last_clean":"2024-04-26T21:08:39.711189+0000","last_became_active":"2024-04-26T21:08:38.217228+0000","last_became_peered":"2024-04-26T21:08:38.217228+0000","last_unstale":"2024-04-26T21:08:39.711189+0000","last_undegraded":"2024-04-26T21:08:39.711189+0000","last_fullsized":"2024-04-26T21:08:39.711189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:00.305226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.216714+0000","last_change":"2024-04-26T21:08:37.170243+0000","last_active":"2024-04-26T21:08:39.216714+0000","last_peered":"2024-04-26T21:08:39.216714+0000","last_clean":"2024-04-26T21:08:39.216714+0000","last_became_active":"2024-04-26T21:08:37.169995+0000","last_became_peered":"2024-04-26T21:08:37.169995+0000","last_unstale":"2024-04-26T21:08:39.216714+0000","last_undegraded":"2024-04-26T21:08:39.216714+0000","last_fullsized":"2024-04-26T21:08:39.216714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:18.839848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.216725+0000","last_change":"2024-04-26T21:08:38.200818+0000","last_active":"2024-04-26T21:08:39.216725+0000","last_peered":"2024-04-26T21:08:39.216725+0000","last_clean":"2024-04-26T21:08:39.216725+0000","last_became_active":"2024-04-26T21:08:38.198256+0000","last_became_peered":"2024-04-26T21:08:38.198256+0000","last_unstale":"2024-04-26T21:08:39.216725+0000","last_undegraded":"2024-04-26T21:08:39.216725+0000","last_fullsized":"2024-04-26T21:08:39.216725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:01.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.191066+0000","last_change":"2024-04-26T21:08:37.170807+0000","last_active":"2024-04-26T21:08:39.191066+0000","last_peered":"2024-04-26T21:08:39.191066+0000","last_clean":"2024-04-26T21:08:39.191066+0000","last_became_active":"2024-04-26T21:08:37.170428+0000","last_became_peered":"2024-04-26T21:08:37.170428+0000","last_unstale":"2024-04-26T21:08:39.191066+0000","last_undegraded":"2024-04-26T21:08:39.191066+0000","last_fullsized":"2024-04-26T21:08:39.191066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:34.683220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711663+0000","last_change":"2024-04-26T21:08:38.209564+0000","last_active":"2024-04-26T21:08:39.711663+0000","last_peered":"2024-04-26T21:08:39.711663+0000","last_clean":"2024-04-26T21:08:39.711663+0000","last_became_active":"2024-04-26T21:08:38.209194+0000","last_became_peered":"2024-04-26T21:08:38.209194+0000","last_unstale":"2024-04-26T21:08:39.711663+0000","last_undegraded":"2024-04-26T21:08:39.711663+0000","last_fullsized":"2024-04-26T21:08:39.711663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:35.132498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.196061+0000","last_change":"2024-04-26T21:08:37.156599+0000","last_active":"2024-04-26T21:08:39.196061+0000","last_peered":"2024-04-26T21:08:39.196061+0000","last_clean":"2024-04-26T21:08:39.196061+0000","last_became_active":"2024-04-26T21:08:37.154195+0000","last_became_peered":"2024-04-26T21:08:37.154195+0000","last_unstale":"2024-04-26T21:08:39.196061+0000","last_undegraded":"2024-04-26T21:08:39.196061+0000","last_fullsized":"2024-04-26T21:08:39.196061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:00.067702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711404+0000","last_change":"2024-04-26T21:08:38.209445+0000","last_active":"2024-04-26T21:08:39.711404+0000","last_peered":"2024-04-26T21:08:39.711404+0000","last_clean":"2024-04-26T21:08:39.711404+0000","last_became_active":"2024-04-26T21:08:38.209134+0000","last_became_peered":"2024-04-26T21:08:38.209134+0000","last_unstale":"2024-04-26T21:08:39.711404+0000","last_undegraded":"2024-04-26T21:08:39.711404+0000","last_fullsized":"2024-04-26T21:08:39.711404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:44.324217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.191281+0000","last_change":"2024-04-26T21:08:37.176762+0000","last_active":"2024-04-26T21:08:39.191281+0000","last_peered":"2024-04-26T21:08:39.191281+0000","last_clean":"2024-04-26T21:08:39.191281+0000","last_became_active":"2024-04-26T21:08:37.176369+0000","last_became_peered":"2024-04-26T21:08:37.176369+0000","last_unstale":"2024-04-26T21:08:39.191281+0000","last_undegraded":"2024-04-26T21:08:39.191281+0000","last_fullsized":"2024-04-26T21:08:39.191281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:00.488332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712217+0000","last_change":"2024-04-26T21:08:38.211784+0000","last_active":"2024-04-26T21:08:39.712217+0000","last_peered":"2024-04-26T21:08:39.712217+0000","last_clean":"2024-04-26T21:08:39.712217+0000","last_became_active":"2024-04-26T21:08:38.210121+0000","last_became_peered":"2024-04-26T21:08:38.210121+0000","last_unstale":"2024-04-26T21:08:39.712217+0000","last_undegraded":"2024-04-26T21:08:39.712217+0000","last_fullsized":"2024-04-26T21:08:39.712217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:46.194523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211130+0000","last_change":"2024-04-26T21:08:37.167350+0000","last_active":"2024-04-26T21:08:39.211130+0000","last_peered":"2024-04-26T21:08:39.211130+0000","last_clean":"2024-04-26T21:08:39.211130+0000","last_became_active":"2024-04-26T21:08:37.166952+0000","last_became_peered":"2024-04-26T21:08:37.166952+0000","last_unstale":"2024-04-26T21:08:39.211130+0000","last_undegraded":"2024-04-26T21:08:39.211130+0000","last_fullsized":"2024-04-26T21:08:39.211130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:30.628461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195797+0000","last_change":"2024-04-26T21:08:38.202682+0000","last_active":"2024-04-26T21:08:39.195797+0000","last_peered":"2024-04-26T21:08:39.195797+0000","last_clean":"2024-04-26T21:08:39.195797+0000","last_became_active":"2024-04-26T21:08:38.202299+0000","last_became_peered":"2024-04-26T21:08:38.202299+0000","last_unstale":"2024-04-26T21:08:39.195797+0000","last_undegraded":"2024-04-26T21:08:39.195797+0000","last_fullsized":"2024-04-26T21:08:39.195797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:19.756896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.191430+0000","last_change":"2024-04-26T21:08:37.168687+0000","last_active":"2024-04-26T21:08:39.191430+0000","last_peered":"2024-04-26T21:08:39.191430+0000","last_clean":"2024-04-26T21:08:39.191430+0000","last_became_active":"2024-04-26T21:08:37.168302+0000","last_became_peered":"2024-04-26T21:08:37.168302+0000","last_unstale":"2024-04-26T21:08:39.191430+0000","last_undegraded":"2024-04-26T21:08:39.191430+0000","last_fullsized":"2024-04-26T21:08:39.191430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:48:02.545851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711826+0000","last_change":"2024-04-26T21:08:38.208610+0000","last_active":"2024-04-26T21:08:39.711826+0000","last_peered":"2024-04-26T21:08:39.711826+0000","last_clean":"2024-04-26T21:08:39.711826+0000","last_became_active":"2024-04-26T21:08:38.206735+0000","last_became_peered":"2024-04-26T21:08:38.206735+0000","last_unstale":"2024-04-26T21:08:39.711826+0000","last_undegraded":"2024-04-26T21:08:39.711826+0000","last_fullsized":"2024-04-26T21:08:39.711826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:54.952928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.710746+0000","last_change":"2024-04-26T21:08:33.504911+0000","last_active":"2024-04-26T21:08:39.710746+0000","last_peered":"2024-04-26T21:08:39.710746+0000","last_clean":"2024-04-26T21:08:39.710746+0000","last_became_active":"2024-04-26T21:08:33.503667+0000","last_became_peered":"2024-04-26T21:08:33.503667+0000","last_unstale":"2024-04-26T21:08:39.710746+0000","last_undegraded":"2024-04-26T21:08:39.710746+0000","last_fullsized":"2024-04-26T21:08:39.710746+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:32.013907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:32.013907+0000","last_clean_scrub_stamp":"2024-04-26T21:08:32.013907+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:09.596153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194166+0000","last_change":"2024-04-26T21:08:37.161497+0000","last_active":"2024-04-26T21:08:39.194166+0000","last_peered":"2024-04-26T21:08:39.194166+0000","last_clean":"2024-04-26T21:08:39.194166+0000","last_became_active":"2024-04-26T21:08:37.161144+0000","last_became_peered":"2024-04-26T21:08:37.161144+0000","last_unstale":"2024-04-26T21:08:39.194166+0000","last_undegraded":"2024-04-26T21:08:39.194166+0000","last_fullsized":"2024-04-26T21:08:39.194166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:18.839848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712097+0000","last_change":"2024-04-26T21:08:38.211227+0000","last_active":"2024-04-26T21:08:39.712097+0000","last_peered":"2024-04-26T21:08:39.712097+0000","last_clean":"2024-04-26T21:08:39.712097+0000","last_became_active":"2024-04-26T21:08:38.210882+0000","last_became_peered":"2024-04-26T21:08:38.210882+0000","last_unstale":"2024-04-26T21:08:39.712097+0000","last_undegraded":"2024-04-26T21:08:39.712097+0000","last_fullsized":"2024-04-26T21:08:39.712097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:42.506444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.710561+0000","last_change":"2024-04-26T21:08:37.177216+0000","last_active":"2024-04-26T21:08:39.710561+0000","last_peered":"2024-04-26T21:08:39.710561+0000","last_clean":"2024-04-26T21:08:39.710561+0000","last_became_active":"2024-04-26T21:08:37.176919+0000","last_became_peered":"2024-04-26T21:08:37.176919+0000","last_unstale":"2024-04-26T21:08:39.710561+0000","last_undegraded":"2024-04-26T21:08:39.710561+0000","last_fullsized":"2024-04-26T21:08:39.710561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:58.240486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712236+0000","last_change":"2024-04-26T21:08:38.191994+0000","last_active":"2024-04-26T21:08:39.712236+0000","last_peered":"2024-04-26T21:08:39.712236+0000","last_clean":"2024-04-26T21:08:39.712236+0000","last_became_active":"2024-04-26T21:08:38.191567+0000","last_became_peered":"2024-04-26T21:08:38.191567+0000","last_unstale":"2024-04-26T21:08:39.712236+0000","last_undegraded":"2024-04-26T21:08:39.712236+0000","last_fullsized":"2024-04-26T21:08:39.712236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:01.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.191022+0000","last_change":"2024-04-26T21:08:37.172777+0000","last_active":"2024-04-26T21:08:39.191022+0000","last_peered":"2024-04-26T21:08:39.191022+0000","last_clean":"2024-04-26T21:08:39.191022+0000","last_became_active":"2024-04-26T21:08:37.171865+0000","last_became_peered":"2024-04-26T21:08:37.171865+0000","last_unstale":"2024-04-26T21:08:39.191022+0000","last_undegraded":"2024-04-26T21:08:39.191022+0000","last_fullsized":"2024-04-26T21:08:39.191022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:18.839848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.210976+0000","last_change":"2024-04-26T21:08:38.200547+0000","last_active":"2024-04-26T21:08:39.210976+0000","last_peered":"2024-04-26T21:08:39.210976+0000","last_clean":"2024-04-26T21:08:39.210976+0000","last_became_active":"2024-04-26T21:08:38.200025+0000","last_became_peered":"2024-04-26T21:08:38.200025+0000","last_unstale":"2024-04-26T21:08:39.210976+0000","last_undegraded":"2024-04-26T21:08:39.210976+0000","last_fullsized":"2024-04-26T21:08:39.210976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:59.807950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194881+0000","last_change":"2024-04-26T21:08:37.170761+0000","last_active":"2024-04-26T21:08:39.194881+0000","last_peered":"2024-04-26T21:08:39.194881+0000","last_clean":"2024-04-26T21:08:39.194881+0000","last_became_active":"2024-04-26T21:08:37.170431+0000","last_became_peered":"2024-04-26T21:08:37.170431+0000","last_unstale":"2024-04-26T21:08:39.194881+0000","last_undegraded":"2024-04-26T21:08:39.194881+0000","last_fullsized":"2024-04-26T21:08:39.194881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:59.977930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195387+0000","last_change":"2024-04-26T21:08:38.213352+0000","last_active":"2024-04-26T21:08:39.195387+0000","last_peered":"2024-04-26T21:08:39.195387+0000","last_clean":"2024-04-26T21:08:39.195387+0000","last_became_active":"2024-04-26T21:08:38.201057+0000","last_became_peered":"2024-04-26T21:08:38.201057+0000","last_unstale":"2024-04-26T21:08:39.195387+0000","last_undegraded":"2024-04-26T21:08:39.195387+0000","last_fullsized":"2024-04-26T21:08:39.195387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:35.698263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.210868+0000","last_change":"2024-04-26T21:08:37.169249+0000","last_active":"2024-04-26T21:08:39.210868+0000","last_peered":"2024-04-26T21:08:39.210868+0000","last_clean":"2024-04-26T21:08:39.210868+0000","last_became_active":"2024-04-26T21:08:37.168910+0000","last_became_peered":"2024-04-26T21:08:37.168910+0000","last_unstale":"2024-04-26T21:08:39.210868+0000","last_undegraded":"2024-04-26T21:08:39.210868+0000","last_fullsized":"2024-04-26T21:08:39.210868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:34.683220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194515+0000","last_change":"2024-04-26T21:08:38.206722+0000","last_active":"2024-04-26T21:08:39.194515+0000","last_peered":"2024-04-26T21:08:39.194515+0000","last_clean":"2024-04-26T21:08:39.194515+0000","last_became_active":"2024-04-26T21:08:38.205394+0000","last_became_peered":"2024-04-26T21:08:38.205394+0000","last_unstale":"2024-04-26T21:08:39.194515+0000","last_undegraded":"2024-04-26T21:08:39.194515+0000","last_fullsized":"2024-04-26T21:08:39.194515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:44.324217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712111+0000","last_change":"2024-04-26T21:08:37.178417+0000","last_active":"2024-04-26T21:08:39.712111+0000","last_peered":"2024-04-26T21:08:39.712111+0000","last_clean":"2024-04-26T21:08:39.712111+0000","last_became_active":"2024-04-26T21:08:37.178031+0000","last_became_peered":"2024-04-26T21:08:37.178031+0000","last_unstale":"2024-04-26T21:08:39.712111+0000","last_undegraded":"2024-04-26T21:08:39.712111+0000","last_fullsized":"2024-04-26T21:08:39.712111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:54.872853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.210824+0000","last_change":"2024-04-26T21:08:38.200358+0000","last_active":"2024-04-26T21:08:39.210824+0000","last_peered":"2024-04-26T21:08:39.210824+0000","last_clean":"2024-04-26T21:08:39.210824+0000","last_became_active":"2024-04-26T21:08:38.200001+0000","last_became_peered":"2024-04-26T21:08:38.200001+0000","last_unstale":"2024-04-26T21:08:39.210824+0000","last_undegraded":"2024-04-26T21:08:39.210824+0000","last_fullsized":"2024-04-26T21:08:39.210824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:01.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.215005+0000","last_change":"2024-04-26T21:08:37.178473+0000","last_active":"2024-04-26T21:08:39.215005+0000","last_peered":"2024-04-26T21:08:39.215005+0000","last_clean":"2024-04-26T21:08:39.215005+0000","last_became_active":"2024-04-26T21:08:37.178079+0000","last_became_peered":"2024-04-26T21:08:37.178079+0000","last_unstale":"2024-04-26T21:08:39.215005+0000","last_undegraded":"2024-04-26T21:08:39.215005+0000","last_fullsized":"2024-04-26T21:08:39.215005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:31.424853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712508+0000","last_change":"2024-04-26T21:08:38.212830+0000","last_active":"2024-04-26T21:08:39.712508+0000","last_peered":"2024-04-26T21:08:39.712508+0000","last_clean":"2024-04-26T21:08:39.712508+0000","last_became_active":"2024-04-26T21:08:38.212327+0000","last_became_peered":"2024-04-26T21:08:38.212327+0000","last_unstale":"2024-04-26T21:08:39.712508+0000","last_undegraded":"2024-04-26T21:08:39.712508+0000","last_fullsized":"2024-04-26T21:08:39.712508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:26.983477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217434+0000","last_change":"2024-04-26T21:08:37.172503+0000","last_active":"2024-04-26T21:08:39.217434+0000","last_peered":"2024-04-26T21:08:39.217434+0000","last_clean":"2024-04-26T21:08:39.217434+0000","last_became_active":"2024-04-26T21:08:37.172129+0000","last_became_peered":"2024-04-26T21:08:37.172129+0000","last_unstale":"2024-04-26T21:08:39.217434+0000","last_undegraded":"2024-04-26T21:08:39.217434+0000","last_fullsized":"2024-04-26T21:08:39.217434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:34.683220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712025+0000","last_change":"2024-04-26T21:08:38.192302+0000","last_active":"2024-04-26T21:08:39.712025+0000","last_peered":"2024-04-26T21:08:39.712025+0000","last_clean":"2024-04-26T21:08:39.712025+0000","last_became_active":"2024-04-26T21:08:38.191741+0000","last_became_peered":"2024-04-26T21:08:38.191741+0000","last_unstale":"2024-04-26T21:08:39.712025+0000","last_undegraded":"2024-04-26T21:08:39.712025+0000","last_fullsized":"2024-04-26T21:08:39.712025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:48.443740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.710597+0000","last_change":"2024-04-26T21:08:37.178123+0000","last_active":"2024-04-26T21:08:39.710597+0000","last_peered":"2024-04-26T21:08:39.710597+0000","last_clean":"2024-04-26T21:08:39.710597+0000","last_became_active":"2024-04-26T21:08:37.177831+0000","last_became_peered":"2024-04-26T21:08:37.177831+0000","last_unstale":"2024-04-26T21:08:39.710597+0000","last_undegraded":"2024-04-26T21:08:39.710597+0000","last_fullsized":"2024-04-26T21:08:39.710597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:59.977930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.196679+0000","last_change":"2024-04-26T21:08:38.212239+0000","last_active":"2024-04-26T21:08:39.196679+0000","last_peered":"2024-04-26T21:08:39.196679+0000","last_clean":"2024-04-26T21:08:39.196679+0000","last_became_active":"2024-04-26T21:08:38.209717+0000","last_became_peered":"2024-04-26T21:08:38.209717+0000","last_unstale":"2024-04-26T21:08:39.196679+0000","last_undegraded":"2024-04-26T21:08:39.196679+0000","last_fullsized":"2024-04-26T21:08:39.196679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:31.643503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211322+0000","last_change":"2024-04-26T21:08:37.173887+0000","last_active":"2024-04-26T21:08:39.211322+0000","last_peered":"2024-04-26T21:08:39.211322+0000","last_clean":"2024-04-26T21:08:39.211322+0000","last_became_active":"2024-04-26T21:08:37.173034+0000","last_became_peered":"2024-04-26T21:08:37.173034+0000","last_unstale":"2024-04-26T21:08:39.211322+0000","last_undegraded":"2024-04-26T21:08:39.211322+0000","last_fullsized":"2024-04-26T21:08:39.211322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:00.488332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712910+0000","last_change":"2024-04-26T21:08:38.197901+0000","last_active":"2024-04-26T21:08:39.712910+0000","last_peered":"2024-04-26T21:08:39.712910+0000","last_clean":"2024-04-26T21:08:39.712910+0000","last_became_active":"2024-04-26T21:08:38.197592+0000","last_became_peered":"2024-04-26T21:08:38.197592+0000","last_unstale":"2024-04-26T21:08:39.712910+0000","last_undegraded":"2024-04-26T21:08:39.712910+0000","last_fullsized":"2024-04-26T21:08:39.712910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:46.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211038+0000","last_change":"2024-04-26T21:08:37.175903+0000","last_active":"2024-04-26T21:08:39.211038+0000","last_peered":"2024-04-26T21:08:39.211038+0000","last_clean":"2024-04-26T21:08:39.211038+0000","last_became_active":"2024-04-26T21:08:37.175605+0000","last_became_peered":"2024-04-26T21:08:37.175605+0000","last_unstale":"2024-04-26T21:08:39.211038+0000","last_undegraded":"2024-04-26T21:08:39.211038+0000","last_fullsized":"2024-04-26T21:08:39.211038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:59.977930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711599+0000","last_change":"2024-04-26T21:08:38.208922+0000","last_active":"2024-04-26T21:08:39.711599+0000","last_peered":"2024-04-26T21:08:39.711599+0000","last_clean":"2024-04-26T21:08:39.711599+0000","last_became_active":"2024-04-26T21:08:38.208597+0000","last_became_peered":"2024-04-26T21:08:38.208597+0000","last_unstale":"2024-04-26T21:08:39.711599+0000","last_undegraded":"2024-04-26T21:08:39.711599+0000","last_fullsized":"2024-04-26T21:08:39.711599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:59.807950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711492+0000","last_change":"2024-04-26T21:08:37.173916+0000","last_active":"2024-04-26T21:08:39.711492+0000","last_peered":"2024-04-26T21:08:39.711492+0000","last_clean":"2024-04-26T21:08:39.711492+0000","last_became_active":"2024-04-26T21:08:37.173553+0000","last_became_peered":"2024-04-26T21:08:37.173553+0000","last_unstale":"2024-04-26T21:08:39.711492+0000","last_undegraded":"2024-04-26T21:08:39.711492+0000","last_fullsized":"2024-04-26T21:08:39.711492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:30.628461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217095+0000","last_change":"2024-04-26T21:08:38.212699+0000","last_active":"2024-04-26T21:08:39.217095+0000","last_peered":"2024-04-26T21:08:39.217095+0000","last_clean":"2024-04-26T21:08:39.217095+0000","last_became_active":"2024-04-26T21:08:38.212418+0000","last_became_peered":"2024-04-26T21:08:38.212418+0000","last_unstale":"2024-04-26T21:08:39.217095+0000","last_undegraded":"2024-04-26T21:08:39.217095+0000","last_fullsized":"2024-04-26T21:08:39.217095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:59.807950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194366+0000","last_change":"2024-04-26T21:08:38.211036+0000","last_active":"2024-04-26T21:08:39.194366+0000","last_peered":"2024-04-26T21:08:39.194366+0000","last_clean":"2024-04-26T21:08:39.194366+0000","last_became_active":"2024-04-26T21:08:38.207129+0000","last_became_peered":"2024-04-26T21:08:38.207129+0000","last_unstale":"2024-04-26T21:08:39.194366+0000","last_undegraded":"2024-04-26T21:08:39.194366+0000","last_fullsized":"2024-04-26T21:08:39.194366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:46.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711252+0000","last_change":"2024-04-26T21:08:37.179811+0000","last_active":"2024-04-26T21:08:39.711252+0000","last_peered":"2024-04-26T21:08:39.711252+0000","last_clean":"2024-04-26T21:08:39.711252+0000","last_became_active":"2024-04-26T21:08:37.179425+0000","last_became_peered":"2024-04-26T21:08:37.179425+0000","last_unstale":"2024-04-26T21:08:39.711252+0000","last_undegraded":"2024-04-26T21:08:39.711252+0000","last_fullsized":"2024-04-26T21:08:39.711252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:31.424853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195669+0000","last_change":"2024-04-26T21:08:38.207278+0000","last_active":"2024-04-26T21:08:39.195669+0000","last_peered":"2024-04-26T21:08:39.195669+0000","last_clean":"2024-04-26T21:08:39.195669+0000","last_became_active":"2024-04-26T21:08:38.204849+0000","last_became_peered":"2024-04-26T21:08:38.204849+0000","last_unstale":"2024-04-26T21:08:39.195669+0000","last_undegraded":"2024-04-26T21:08:39.195669+0000","last_fullsized":"2024-04-26T21:08:39.195669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:54.952928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711653+0000","last_change":"2024-04-26T21:08:37.175992+0000","last_active":"2024-04-26T21:08:39.711653+0000","last_peered":"2024-04-26T21:08:39.711653+0000","last_clean":"2024-04-26T21:08:39.711653+0000","last_became_active":"2024-04-26T21:08:37.175682+0000","last_became_peered":"2024-04-26T21:08:37.175682+0000","last_unstale":"2024-04-26T21:08:39.711653+0000","last_undegraded":"2024-04-26T21:08:39.711653+0000","last_fullsized":"2024-04-26T21:08:39.711653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:33.374984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.710839+0000","last_change":"2024-04-26T21:08:38.207957+0000","last_active":"2024-04-26T21:08:39.710839+0000","last_peered":"2024-04-26T21:08:39.710839+0000","last_clean":"2024-04-26T21:08:39.710839+0000","last_became_active":"2024-04-26T21:08:38.207573+0000","last_became_peered":"2024-04-26T21:08:38.207573+0000","last_unstale":"2024-04-26T21:08:39.710839+0000","last_undegraded":"2024-04-26T21:08:39.710839+0000","last_fullsized":"2024-04-26T21:08:39.710839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:02:14.307645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712093+0000","last_change":"2024-04-26T21:08:37.173718+0000","last_active":"2024-04-26T21:08:39.712093+0000","last_peered":"2024-04-26T21:08:39.712093+0000","last_clean":"2024-04-26T21:08:39.712093+0000","last_became_active":"2024-04-26T21:08:37.170846+0000","last_became_peered":"2024-04-26T21:08:37.170846+0000","last_unstale":"2024-04-26T21:08:39.712093+0000","last_undegraded":"2024-04-26T21:08:39.712093+0000","last_fullsized":"2024-04-26T21:08:39.712093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:48:02.545851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711803+0000","last_change":"2024-04-26T21:08:38.199029+0000","last_active":"2024-04-26T21:08:39.711803+0000","last_peered":"2024-04-26T21:08:39.711803+0000","last_clean":"2024-04-26T21:08:39.711803+0000","last_became_active":"2024-04-26T21:08:38.198720+0000","last_became_peered":"2024-04-26T21:08:38.198720+0000","last_unstale":"2024-04-26T21:08:39.711803+0000","last_undegraded":"2024-04-26T21:08:39.711803+0000","last_fullsized":"2024-04-26T21:08:39.711803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:40.618646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712817+0000","last_change":"2024-04-26T21:08:37.165057+0000","last_active":"2024-04-26T21:08:39.712817+0000","last_peered":"2024-04-26T21:08:39.712817+0000","last_clean":"2024-04-26T21:08:39.712817+0000","last_became_active":"2024-04-26T21:08:37.164781+0000","last_became_peered":"2024-04-26T21:08:37.164781+0000","last_unstale":"2024-04-26T21:08:39.712817+0000","last_undegraded":"2024-04-26T21:08:39.712817+0000","last_fullsized":"2024-04-26T21:08:39.712817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:17.499100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:08:42.635 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:08:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.871+0000 7fbd42400640 1 Processor -- start 2024-04-26T21:08:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.872+0000 7fbd42400640 1 -- start start 2024-04-26T21:08:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.873+0000 7fbd42400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbd3c0d1c80 0x7fbd3c0d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.873+0000 7fbd42400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbd3c0d2060 0x7fbd3c162490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.873+0000 7fbd42400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd3c162e60 con 0x7fbd3c1629d0 2024-04-26T21:08:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.873+0000 7fbd42400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fbd3c0b5f70 con 0x7fbd3c0d2060 2024-04-26T21:08:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.873+0000 7fbd42400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbd3c0d3830 con 0x7fbd3c0d1c80 2024-04-26T21:08:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.873+0000 7fbd3be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbd3c0d2060 0x7fbd3c162490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.874+0000 7fbd3be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbd3c0d2060 0x7fbd3c162490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34468/0 (socket says 172.21.15.26:34468) 2024-04-26T21:08:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.874+0000 7fbd3be00640 1 -- 172.21.15.26:0/4076734657 learned_addr learned my addr 172.21.15.26:0/4076734657 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.874+0000 7fbd3be00640 1 -- 172.21.15.26:0/4076734657 >> v2:172.21.15.86:3300/0 conn(0x7fbd3c0d1c80 msgr2=0x7fbd3c0d3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.874+0000 7fbd3be00640 1 --2- 172.21.15.26:0/4076734657 >> v2:172.21.15.86:3300/0 conn(0x7fbd3c0d1c80 0x7fbd3c0d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.874+0000 7fbd3be00640 1 -- 172.21.15.26:0/4076734657 >> v1:172.21.15.86:6789/0 conn(0x7fbd3c1629d0 legacy=0x7fbd3c0c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.874+0000 7fbd3be00640 1 -- 172.21.15.26:0/4076734657 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3c0b61c0 con 0x7fbd3c0d2060 2024-04-26T21:08:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.875+0000 7fbd3be00640 1 --2- 172.21.15.26:0/4076734657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd3c0d2060 0x7fbd3c162490 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c08d3b0 tx=0x7fbd2802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2c7579d8f9c0a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.875+0000 7fbd3b400640 1 -- 172.21.15.26:0/4076734657 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28037070 con 0x7fbd3c0d2060 2024-04-26T21:08:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.875+0000 7fbd3b400640 1 -- 172.21.15.26:0/4076734657 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd28032440 con 0x7fbd3c0d2060 2024-04-26T21:08:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.875+0000 7fbd3b400640 1 -- 172.21.15.26:0/4076734657 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28032810 con 0x7fbd3c0d2060 2024-04-26T21:08:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.876+0000 7fbd42400640 1 -- 172.21.15.26:0/4076734657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd3c0d2060 msgr2=0x7fbd3c162490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.876+0000 7fbd42400640 1 --2- 172.21.15.26:0/4076734657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd3c0d2060 0x7fbd3c162490 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c08d3b0 tx=0x7fbd2802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.876+0000 7fbd42400640 1 -- 172.21.15.26:0/4076734657 shutdown_connections 2024-04-26T21:08:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.876+0000 7fbd42400640 1 --2- 172.21.15.26:0/4076734657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd3c0d2060 0x7fbd3c162490 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.876+0000 7fbd42400640 1 --2- 172.21.15.26:0/4076734657 >> v2:172.21.15.86:3300/0 conn(0x7fbd3c0d1c80 0x7fbd3c0d3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.876+0000 7fbd42400640 1 -- 172.21.15.26:0/4076734657 >> 172.21.15.26:0/4076734657 conn(0x7fbd3c0d5020 msgr2=0x7fbd3c0d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.876+0000 7fbd42400640 1 -- 172.21.15.26:0/4076734657 shutdown_connections 2024-04-26T21:08:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.876+0000 7fbd42400640 1 -- 172.21.15.26:0/4076734657 wait complete. 2024-04-26T21:08:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.878+0000 7fbd42400640 1 Processor -- start 2024-04-26T21:08:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.878+0000 7fbd42400640 1 -- start start 2024-04-26T21:08:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.879+0000 7fbd42400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd3c0d1c80 0x7fbd3c154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.879+0000 7fbd42400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd3c0d2060 0x7fbd3c158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.879+0000 7fbd42400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3c1629d0 0x7fbd3c0cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.879+0000 7fbd42400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c074790 con 0x7fbd3c0d1c80 2024-04-26T21:08:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.879+0000 7fbd42400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c074610 con 0x7fbd3c1629d0 2024-04-26T21:08:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.879+0000 7fbd42400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbd3c0d5b40 con 0x7fbd3c0d2060 2024-04-26T21:08:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.879+0000 7fbd41000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd3c0d1c80 0x7fbd3c154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.879+0000 7fbd41000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd3c0d1c80 0x7fbd3c154380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34474/0 (socket says 172.21.15.26:34474) 2024-04-26T21:08:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.879+0000 7fbd41000640 1 -- 172.21.15.26:0/3638878248 learned_addr learned my addr 172.21.15.26:0/3638878248 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.879+0000 7fbd41a00640 1 --2- 172.21.15.26:0/3638878248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3c1629d0 0x7fbd3c0cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.879+0000 7fbd41000640 1 -- 172.21.15.26:0/3638878248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd3c0d2060 msgr2=0x7fbd3c158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.879+0000 7fbd3be00640 1 --2- 172.21.15.26:0/3638878248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd3c0d2060 0x7fbd3c158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.879+0000 7fbd41000640 1 --2- 172.21.15.26:0/3638878248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd3c0d2060 0x7fbd3c158ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.879+0000 7fbd41000640 1 -- 172.21.15.26:0/3638878248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3c1629d0 msgr2=0x7fbd3c0cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.879+0000 7fbd41000640 1 --2- 172.21.15.26:0/3638878248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3c1629d0 0x7fbd3c0cafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.879+0000 7fbd41000640 1 -- 172.21.15.26:0/3638878248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3c0c9f20 con 0x7fbd3c0d1c80 2024-04-26T21:08:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.880+0000 7fbd41000640 1 --2- 172.21.15.26:0/3638878248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd3c0d1c80 0x7fbd3c154380 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c00ba50 tx=0x7fbd2c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.880+0000 7fbd38c00640 1 -- 172.21.15.26:0/3638878248 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c019070 con 0x7fbd3c0d1c80 2024-04-26T21:08:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.880+0000 7fbd38c00640 1 -- 172.21.15.26:0/3638878248 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd2c00cc00 con 0x7fbd3c0d1c80 2024-04-26T21:08:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.880+0000 7fbd38c00640 1 -- 172.21.15.26:0/3638878248 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c014440 con 0x7fbd3c0d1c80 2024-04-26T21:08:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.880+0000 7fbd42400640 1 -- 172.21.15.26:0/3638878248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd3c0ca200 con 0x7fbd3c0d1c80 2024-04-26T21:08:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.880+0000 7fbd42400640 1 -- 172.21.15.26:0/3638878248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd3c0ca7c0 con 0x7fbd3c0d1c80 2024-04-26T21:08:42.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.881+0000 7fbd42400640 1 -- 172.21.15.26:0/3638878248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd3c06a700 con 0x7fbd3c0d1c80 2024-04-26T21:08:42.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.882+0000 7fbd38c00640 1 -- 172.21.15.26:0/3638878248 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd2c01c070 con 0x7fbd3c0d1c80 2024-04-26T21:08:42.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.884+0000 7fbd38c00640 1 --2- 172.21.15.26:0/3638878248 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fbd1c06c8f0 0x7fbd1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:42.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.884+0000 7fbd38c00640 1 -- 172.21.15.26:0/3638878248 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbd2c08b980 con 0x7fbd3c0d1c80 2024-04-26T21:08:42.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.884+0000 7fbd3be00640 1 --2- 172.21.15.26:0/3638878248 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fbd1c06c8f0 0x7fbd1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:42.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.884+0000 7fbd3be00640 1 --2- 172.21.15.26:0/3638878248 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fbd1c06c8f0 0x7fbd1c06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbd28034000 tx=0x7fbd2802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:42.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:42.888+0000 7fbd38c00640 1 -- 172.21.15.26:0/3638878248 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbd2c059d10 con 0x7fbd3c0d1c80 2024-04-26T21:08:43.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.089+0000 7fbd42400640 1 -- 172.21.15.26:0/3638878248 --> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbd3c1548c0 con 0x7fbd1c06c8f0 2024-04-26T21:08:43.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.100+0000 7fbd38c00640 1 -- 172.21.15.26:0/3638878248 <== mgr.4104 v2:172.21.15.26:6832/3130430358 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7fbd3c1548c0 con 0x7fbd1c06c8f0 2024-04-26T21:08:43.104 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:43.105 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-26T21:08:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.105+0000 7fbd42400640 1 -- 172.21.15.26:0/3638878248 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fbd1c06c8f0 msgr2=0x7fbd1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.105+0000 7fbd42400640 1 --2- 172.21.15.26:0/3638878248 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fbd1c06c8f0 0x7fbd1c06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbd28034000 tx=0x7fbd2802e000 comp rx=0 tx=0).stop 2024-04-26T21:08:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.105+0000 7fbd42400640 1 -- 172.21.15.26:0/3638878248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd3c0d1c80 msgr2=0x7fbd3c154380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.105+0000 7fbd42400640 1 --2- 172.21.15.26:0/3638878248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd3c0d1c80 0x7fbd3c154380 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c00ba50 tx=0x7fbd2c00bf10 comp rx=0 tx=0).stop 2024-04-26T21:08:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.105+0000 7fbd42400640 1 -- 172.21.15.26:0/3638878248 shutdown_connections 2024-04-26T21:08:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.105+0000 7fbd42400640 1 --2- 172.21.15.26:0/3638878248 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fbd1c06c8f0 0x7fbd1c06edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:43.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.105+0000 7fbd42400640 1 --2- 172.21.15.26:0/3638878248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3c1629d0 0x7fbd3c0cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:43.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.105+0000 7fbd42400640 1 --2- 172.21.15.26:0/3638878248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd3c0d2060 0x7fbd3c158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:43.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.105+0000 7fbd42400640 1 --2- 172.21.15.26:0/3638878248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd3c0d1c80 0x7fbd3c154380 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:43.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.105+0000 7fbd42400640 1 -- 172.21.15.26:0/3638878248 >> 172.21.15.26:0/3638878248 conn(0x7fbd3c0d5020 msgr2=0x7fbd3c1600d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:43.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.106+0000 7fbd42400640 1 -- 172.21.15.26:0/3638878248 shutdown_connections 2024-04-26T21:08:43.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.106+0000 7fbd42400640 1 -- 172.21.15.26:0/3638878248 wait complete. 2024-04-26T21:08:43.125 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-26T21:08:41.286522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235764,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738956,"statfs":{"total":773094113280,"available":772852690944,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.161313"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711962+0000","last_change":"2024-04-26T21:08:38.198898+0000","last_active":"2024-04-26T21:08:39.711962+0000","last_peered":"2024-04-26T21:08:39.711962+0000","last_clean":"2024-04-26T21:08:39.711962+0000","last_became_active":"2024-04-26T21:08:38.198596+0000","last_became_peered":"2024-04-26T21:08:38.198596+0000","last_unstale":"2024-04-26T21:08:39.711962+0000","last_undegraded":"2024-04-26T21:08:39.711962+0000","last_fullsized":"2024-04-26T21:08:39.711962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:19.756896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211160+0000","last_change":"2024-04-26T21:08:38.194080+0000","last_active":"2024-04-26T21:08:39.211160+0000","last_peered":"2024-04-26T21:08:39.211160+0000","last_clean":"2024-04-26T21:08:39.211160+0000","last_became_active":"2024-04-26T21:08:38.193534+0000","last_became_peered":"2024-04-26T21:08:38.193534+0000","last_unstale":"2024-04-26T21:08:39.211160+0000","last_undegraded":"2024-04-26T21:08:39.211160+0000","last_fullsized":"2024-04-26T21:08:39.211160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:22.132136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712041+0000","last_change":"2024-04-26T21:08:38.198384+0000","last_active":"2024-04-26T21:08:39.712041+0000","last_peered":"2024-04-26T21:08:39.712041+0000","last_clean":"2024-04-26T21:08:39.712041+0000","last_became_active":"2024-04-26T21:08:38.198055+0000","last_became_peered":"2024-04-26T21:08:38.198055+0000","last_unstale":"2024-04-26T21:08:39.712041+0000","last_undegraded":"2024-04-26T21:08:39.712041+0000","last_fullsized":"2024-04-26T21:08:39.712041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:55.566474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194277+0000","last_change":"2024-04-26T21:08:38.205534+0000","last_active":"2024-04-26T21:08:39.194277+0000","last_peered":"2024-04-26T21:08:39.194277+0000","last_clean":"2024-04-26T21:08:39.194277+0000","last_became_active":"2024-04-26T21:08:38.205235+0000","last_became_peered":"2024-04-26T21:08:38.205235+0000","last_unstale":"2024-04-26T21:08:39.194277+0000","last_undegraded":"2024-04-26T21:08:39.194277+0000","last_fullsized":"2024-04-26T21:08:39.194277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:26.983477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711420+0000","last_change":"2024-04-26T21:08:38.200597+0000","last_active":"2024-04-26T21:08:39.711420+0000","last_peered":"2024-04-26T21:08:39.711420+0000","last_clean":"2024-04-26T21:08:39.711420+0000","last_became_active":"2024-04-26T21:08:38.197772+0000","last_became_peered":"2024-04-26T21:08:38.197772+0000","last_unstale":"2024-04-26T21:08:39.711420+0000","last_undegraded":"2024-04-26T21:08:39.711420+0000","last_fullsized":"2024-04-26T21:08:39.711420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:17.370683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.196006+0000","last_change":"2024-04-26T21:08:38.210251+0000","last_active":"2024-04-26T21:08:39.196006+0000","last_peered":"2024-04-26T21:08:39.196006+0000","last_clean":"2024-04-26T21:08:39.196006+0000","last_became_active":"2024-04-26T21:08:38.209628+0000","last_became_peered":"2024-04-26T21:08:38.209628+0000","last_unstale":"2024-04-26T21:08:39.196006+0000","last_undegraded":"2024-04-26T21:08:39.196006+0000","last_fullsized":"2024-04-26T21:08:39.196006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:28.069603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711557+0000","last_change":"2024-04-26T21:08:38.199522+0000","last_active":"2024-04-26T21:08:39.711557+0000","last_peered":"2024-04-26T21:08:39.711557+0000","last_clean":"2024-04-26T21:08:39.711557+0000","last_became_active":"2024-04-26T21:08:38.198122+0000","last_became_peered":"2024-04-26T21:08:38.198122+0000","last_unstale":"2024-04-26T21:08:39.711557+0000","last_undegraded":"2024-04-26T21:08:39.711557+0000","last_fullsized":"2024-04-26T21:08:39.711557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:55.797399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711087+0000","last_change":"2024-04-26T21:08:38.207966+0000","last_active":"2024-04-26T21:08:39.711087+0000","last_peered":"2024-04-26T21:08:39.711087+0000","last_clean":"2024-04-26T21:08:39.711087+0000","last_became_active":"2024-04-26T21:08:38.207617+0000","last_became_peered":"2024-04-26T21:08:38.207617+0000","last_unstale":"2024-04-26T21:08:39.711087+0000","last_undegraded":"2024-04-26T21:08:39.711087+0000","last_fullsized":"2024-04-26T21:08:39.711087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:55.566474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712128+0000","last_change":"2024-04-26T21:08:38.204121+0000","last_active":"2024-04-26T21:08:39.712128+0000","last_peered":"2024-04-26T21:08:39.712128+0000","last_clean":"2024-04-26T21:08:39.712128+0000","last_became_active":"2024-04-26T21:08:38.203125+0000","last_became_peered":"2024-04-26T21:08:38.203125+0000","last_unstale":"2024-04-26T21:08:39.712128+0000","last_undegraded":"2024-04-26T21:08:39.712128+0000","last_fullsized":"2024-04-26T21:08:39.712128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:36.343803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217352+0000","last_change":"2024-04-26T21:08:38.204308+0000","last_active":"2024-04-26T21:08:39.217352+0000","last_peered":"2024-04-26T21:08:39.217352+0000","last_clean":"2024-04-26T21:08:39.217352+0000","last_became_active":"2024-04-26T21:08:38.203942+0000","last_became_peered":"2024-04-26T21:08:38.203942+0000","last_unstale":"2024-04-26T21:08:39.217352+0000","last_undegraded":"2024-04-26T21:08:39.217352+0000","last_fullsized":"2024-04-26T21:08:39.217352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:42.506444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217214+0000","last_change":"2024-04-26T21:08:38.201929+0000","last_active":"2024-04-26T21:08:39.217214+0000","last_peered":"2024-04-26T21:08:39.217214+0000","last_clean":"2024-04-26T21:08:39.217214+0000","last_became_active":"2024-04-26T21:08:38.201646+0000","last_became_peered":"2024-04-26T21:08:38.201646+0000","last_unstale":"2024-04-26T21:08:39.217214+0000","last_undegraded":"2024-04-26T21:08:39.217214+0000","last_fullsized":"2024-04-26T21:08:39.217214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:40.618646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195574+0000","last_change":"2024-04-26T21:08:38.207847+0000","last_active":"2024-04-26T21:08:39.195574+0000","last_peered":"2024-04-26T21:08:39.195574+0000","last_clean":"2024-04-26T21:08:39.195574+0000","last_became_active":"2024-04-26T21:08:38.204908+0000","last_became_peered":"2024-04-26T21:08:38.204908+0000","last_unstale":"2024-04-26T21:08:39.195574+0000","last_undegraded":"2024-04-26T21:08:39.195574+0000","last_fullsized":"2024-04-26T21:08:39.195574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:00.992973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711183+0000","last_change":"2024-04-26T21:08:38.203775+0000","last_active":"2024-04-26T21:08:39.711183+0000","last_peered":"2024-04-26T21:08:39.711183+0000","last_clean":"2024-04-26T21:08:39.711183+0000","last_became_active":"2024-04-26T21:08:38.203139+0000","last_became_peered":"2024-04-26T21:08:38.203139+0000","last_unstale":"2024-04-26T21:08:39.711183+0000","last_undegraded":"2024-04-26T21:08:39.711183+0000","last_fullsized":"2024-04-26T21:08:39.711183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:01.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712480+0000","last_change":"2024-04-26T21:08:38.195656+0000","last_active":"2024-04-26T21:08:39.712480+0000","last_peered":"2024-04-26T21:08:39.712480+0000","last_clean":"2024-04-26T21:08:39.712480+0000","last_became_active":"2024-04-26T21:08:38.195240+0000","last_became_peered":"2024-04-26T21:08:38.195240+0000","last_unstale":"2024-04-26T21:08:39.712480+0000","last_undegraded":"2024-04-26T21:08:39.712480+0000","last_fullsized":"2024-04-26T21:08:39.712480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:16.232014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211031+0000","last_change":"2024-04-26T21:08:38.186523+0000","last_active":"2024-04-26T21:08:39.211031+0000","last_peered":"2024-04-26T21:08:39.211031+0000","last_clean":"2024-04-26T21:08:39.211031+0000","last_became_active":"2024-04-26T21:08:38.186159+0000","last_became_peered":"2024-04-26T21:08:38.186159+0000","last_unstale":"2024-04-26T21:08:39.211031+0000","last_undegraded":"2024-04-26T21:08:39.211031+0000","last_fullsized":"2024-04-26T21:08:39.211031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:40.618646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194033+0000","last_change":"2024-04-26T21:08:38.212958+0000","last_active":"2024-04-26T21:08:39.194033+0000","last_peered":"2024-04-26T21:08:39.194033+0000","last_clean":"2024-04-26T21:08:39.194033+0000","last_became_active":"2024-04-26T21:08:38.212666+0000","last_became_peered":"2024-04-26T21:08:38.212666+0000","last_unstale":"2024-04-26T21:08:39.194033+0000","last_undegraded":"2024-04-26T21:08:39.194033+0000","last_fullsized":"2024-04-26T21:08:39.194033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:42.506444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195541+0000","last_change":"2024-04-26T21:08:38.214751+0000","last_active":"2024-04-26T21:08:39.195541+0000","last_peered":"2024-04-26T21:08:39.195541+0000","last_clean":"2024-04-26T21:08:39.195541+0000","last_became_active":"2024-04-26T21:08:38.212857+0000","last_became_peered":"2024-04-26T21:08:38.212857+0000","last_unstale":"2024-04-26T21:08:39.195541+0000","last_undegraded":"2024-04-26T21:08:39.195541+0000","last_fullsized":"2024-04-26T21:08:39.195541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:35.132498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.191204+0000","last_change":"2024-04-26T21:08:38.198823+0000","last_active":"2024-04-26T21:08:39.191204+0000","last_peered":"2024-04-26T21:08:39.191204+0000","last_clean":"2024-04-26T21:08:39.191204+0000","last_became_active":"2024-04-26T21:08:38.198521+0000","last_became_peered":"2024-04-26T21:08:38.198521+0000","last_unstale":"2024-04-26T21:08:39.191204+0000","last_undegraded":"2024-04-26T21:08:39.191204+0000","last_fullsized":"2024-04-26T21:08:39.191204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:34.390026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712075+0000","last_change":"2024-04-26T21:08:38.200413+0000","last_active":"2024-04-26T21:08:39.712075+0000","last_peered":"2024-04-26T21:08:39.712075+0000","last_clean":"2024-04-26T21:08:39.712075+0000","last_became_active":"2024-04-26T21:08:38.200023+0000","last_became_peered":"2024-04-26T21:08:38.200023+0000","last_unstale":"2024-04-26T21:08:39.712075+0000","last_undegraded":"2024-04-26T21:08:39.712075+0000","last_fullsized":"2024-04-26T21:08:39.712075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:28.069603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.216976+0000","last_change":"2024-04-26T21:08:38.211354+0000","last_active":"2024-04-26T21:08:39.216976+0000","last_peered":"2024-04-26T21:08:39.216976+0000","last_clean":"2024-04-26T21:08:39.216976+0000","last_became_active":"2024-04-26T21:08:38.211016+0000","last_became_peered":"2024-04-26T21:08:38.211016+0000","last_unstale":"2024-04-26T21:08:39.216976+0000","last_undegraded":"2024-04-26T21:08:39.216976+0000","last_fullsized":"2024-04-26T21:08:39.216976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:29.659217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712867+0000","last_change":"2024-04-26T21:08:38.201685+0000","last_active":"2024-04-26T21:08:39.712867+0000","last_peered":"2024-04-26T21:08:39.712867+0000","last_clean":"2024-04-26T21:08:39.712867+0000","last_became_active":"2024-04-26T21:08:38.200467+0000","last_became_peered":"2024-04-26T21:08:38.200467+0000","last_unstale":"2024-04-26T21:08:39.712867+0000","last_undegraded":"2024-04-26T21:08:39.712867+0000","last_fullsized":"2024-04-26T21:08:39.712867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:40.618646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194361+0000","last_change":"2024-04-26T21:08:38.204033+0000","last_active":"2024-04-26T21:08:39.194361+0000","last_peered":"2024-04-26T21:08:39.194361+0000","last_clean":"2024-04-26T21:08:39.194361+0000","last_became_active":"2024-04-26T21:08:38.202425+0000","last_became_peered":"2024-04-26T21:08:38.202425+0000","last_unstale":"2024-04-26T21:08:39.194361+0000","last_undegraded":"2024-04-26T21:08:39.194361+0000","last_fullsized":"2024-04-26T21:08:39.194361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:00.305226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711233+0000","last_change":"2024-04-26T21:08:38.216188+0000","last_active":"2024-04-26T21:08:39.711233+0000","last_peered":"2024-04-26T21:08:39.711233+0000","last_clean":"2024-04-26T21:08:39.711233+0000","last_became_active":"2024-04-26T21:08:38.215866+0000","last_became_peered":"2024-04-26T21:08:38.215866+0000","last_unstale":"2024-04-26T21:08:39.711233+0000","last_undegraded":"2024-04-26T21:08:39.711233+0000","last_fullsized":"2024-04-26T21:08:39.711233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:29.659217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.193946+0000","last_change":"2024-04-26T21:08:38.213795+0000","last_active":"2024-04-26T21:08:39.193946+0000","last_peered":"2024-04-26T21:08:39.193946+0000","last_clean":"2024-04-26T21:08:39.193946+0000","last_became_active":"2024-04-26T21:08:38.211952+0000","last_became_peered":"2024-04-26T21:08:38.211952+0000","last_unstale":"2024-04-26T21:08:39.193946+0000","last_undegraded":"2024-04-26T21:08:39.193946+0000","last_fullsized":"2024-04-26T21:08:39.193946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:22.132136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712055+0000","last_change":"2024-04-26T21:08:38.211961+0000","last_active":"2024-04-26T21:08:39.712055+0000","last_peered":"2024-04-26T21:08:39.712055+0000","last_clean":"2024-04-26T21:08:39.712055+0000","last_became_active":"2024-04-26T21:08:38.211590+0000","last_became_peered":"2024-04-26T21:08:38.211590+0000","last_unstale":"2024-04-26T21:08:39.712055+0000","last_undegraded":"2024-04-26T21:08:39.712055+0000","last_fullsized":"2024-04-26T21:08:39.712055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:22.132136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217302+0000","last_change":"2024-04-26T21:08:38.200054+0000","last_active":"2024-04-26T21:08:39.217302+0000","last_peered":"2024-04-26T21:08:39.217302+0000","last_clean":"2024-04-26T21:08:39.217302+0000","last_became_active":"2024-04-26T21:08:38.199738+0000","last_became_peered":"2024-04-26T21:08:38.199738+0000","last_unstale":"2024-04-26T21:08:39.217302+0000","last_undegraded":"2024-04-26T21:08:39.217302+0000","last_fullsized":"2024-04-26T21:08:39.217302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:22.132136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.210875+0000","last_change":"2024-04-26T21:08:38.195577+0000","last_active":"2024-04-26T21:08:39.210875+0000","last_peered":"2024-04-26T21:08:39.210875+0000","last_clean":"2024-04-26T21:08:39.210875+0000","last_became_active":"2024-04-26T21:08:38.195284+0000","last_became_peered":"2024-04-26T21:08:38.195284+0000","last_unstale":"2024-04-26T21:08:39.210875+0000","last_undegraded":"2024-04-26T21:08:39.210875+0000","last_fullsized":"2024-04-26T21:08:39.210875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:29.659217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.713361+0000","last_change":"2024-04-26T21:08:38.202932+0000","last_active":"2024-04-26T21:08:39.713361+0000","last_peered":"2024-04-26T21:08:39.713361+0000","last_clean":"2024-04-26T21:08:39.713361+0000","last_became_active":"2024-04-26T21:08:38.202614+0000","last_became_peered":"2024-04-26T21:08:38.202614+0000","last_unstale":"2024-04-26T21:08:39.713361+0000","last_undegraded":"2024-04-26T21:08:39.713361+0000","last_fullsized":"2024-04-26T21:08:39.713361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:44.324217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.713283+0000","last_change":"2024-04-26T21:08:38.198429+0000","last_active":"2024-04-26T21:08:39.713283+0000","last_peered":"2024-04-26T21:08:39.713283+0000","last_clean":"2024-04-26T21:08:39.713283+0000","last_became_active":"2024-04-26T21:08:38.198144+0000","last_became_peered":"2024-04-26T21:08:38.198144+0000","last_unstale":"2024-04-26T21:08:39.713283+0000","last_undegraded":"2024-04-26T21:08:39.713283+0000","last_fullsized":"2024-04-26T21:08:39.713283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:22.132136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194610+0000","last_change":"2024-04-26T21:08:38.205890+0000","last_active":"2024-04-26T21:08:39.194610+0000","last_peered":"2024-04-26T21:08:39.194610+0000","last_clean":"2024-04-26T21:08:39.194610+0000","last_became_active":"2024-04-26T21:08:38.205583+0000","last_became_peered":"2024-04-26T21:08:38.205583+0000","last_unstale":"2024-04-26T21:08:39.194610+0000","last_undegraded":"2024-04-26T21:08:39.194610+0000","last_fullsized":"2024-04-26T21:08:39.194610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:17.370683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.190845+0000","last_change":"2024-04-26T21:08:38.204162+0000","last_active":"2024-04-26T21:08:39.190845+0000","last_peered":"2024-04-26T21:08:39.190845+0000","last_clean":"2024-04-26T21:08:39.190845+0000","last_became_active":"2024-04-26T21:08:38.203855+0000","last_became_peered":"2024-04-26T21:08:38.203855+0000","last_unstale":"2024-04-26T21:08:39.190845+0000","last_undegraded":"2024-04-26T21:08:39.190845+0000","last_fullsized":"2024-04-26T21:08:39.190845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:31.643503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.216694+0000","last_change":"2024-04-26T21:08:38.207713+0000","last_active":"2024-04-26T21:08:39.216694+0000","last_peered":"2024-04-26T21:08:39.216694+0000","last_clean":"2024-04-26T21:08:39.216694+0000","last_became_active":"2024-04-26T21:08:38.207384+0000","last_became_peered":"2024-04-26T21:08:38.207384+0000","last_unstale":"2024-04-26T21:08:39.216694+0000","last_undegraded":"2024-04-26T21:08:39.216694+0000","last_fullsized":"2024-04-26T21:08:39.216694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:55.566474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.196940+0000","last_change":"2024-04-26T21:08:38.194999+0000","last_active":"2024-04-26T21:08:39.196940+0000","last_peered":"2024-04-26T21:08:39.196940+0000","last_clean":"2024-04-26T21:08:39.196940+0000","last_became_active":"2024-04-26T21:08:38.194647+0000","last_became_peered":"2024-04-26T21:08:38.194647+0000","last_unstale":"2024-04-26T21:08:39.196940+0000","last_undegraded":"2024-04-26T21:08:39.196940+0000","last_fullsized":"2024-04-26T21:08:39.196940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:34.390026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711906+0000","last_change":"2024-04-26T21:08:37.168719+0000","last_active":"2024-04-26T21:08:39.711906+0000","last_peered":"2024-04-26T21:08:39.711906+0000","last_clean":"2024-04-26T21:08:39.711906+0000","last_became_active":"2024-04-26T21:08:37.168257+0000","last_became_peered":"2024-04-26T21:08:37.168257+0000","last_unstale":"2024-04-26T21:08:39.711906+0000","last_undegraded":"2024-04-26T21:08:39.711906+0000","last_fullsized":"2024-04-26T21:08:39.711906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:18.839848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.713404+0000","last_change":"2024-04-26T21:08:38.190358+0000","last_active":"2024-04-26T21:08:39.713404+0000","last_peered":"2024-04-26T21:08:39.713404+0000","last_clean":"2024-04-26T21:08:39.713404+0000","last_became_active":"2024-04-26T21:08:38.189905+0000","last_became_peered":"2024-04-26T21:08:38.189905+0000","last_unstale":"2024-04-26T21:08:39.713404+0000","last_undegraded":"2024-04-26T21:08:39.713404+0000","last_fullsized":"2024-04-26T21:08:39.713404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:42.506444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.216776+0000","last_change":"2024-04-26T21:08:37.177780+0000","last_active":"2024-04-26T21:08:39.216776+0000","last_peered":"2024-04-26T21:08:39.216776+0000","last_clean":"2024-04-26T21:08:39.216776+0000","last_became_active":"2024-04-26T21:08:37.177488+0000","last_became_peered":"2024-04-26T21:08:37.177488+0000","last_unstale":"2024-04-26T21:08:39.216776+0000","last_undegraded":"2024-04-26T21:08:39.216776+0000","last_fullsized":"2024-04-26T21:08:39.216776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:59.977930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711590+0000","last_change":"2024-04-26T21:08:37.171155+0000","last_active":"2024-04-26T21:08:39.711590+0000","last_peered":"2024-04-26T21:08:39.711590+0000","last_clean":"2024-04-26T21:08:39.711590+0000","last_became_active":"2024-04-26T21:08:37.170816+0000","last_became_peered":"2024-04-26T21:08:37.170816+0000","last_unstale":"2024-04-26T21:08:39.711590+0000","last_undegraded":"2024-04-26T21:08:39.711590+0000","last_fullsized":"2024-04-26T21:08:39.711590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:17.499100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712682+0000","last_change":"2024-04-26T21:08:38.206720+0000","last_active":"2024-04-26T21:08:39.712682+0000","last_peered":"2024-04-26T21:08:39.712682+0000","last_clean":"2024-04-26T21:08:39.712682+0000","last_became_active":"2024-04-26T21:08:38.206401+0000","last_became_peered":"2024-04-26T21:08:38.206401+0000","last_unstale":"2024-04-26T21:08:39.712682+0000","last_undegraded":"2024-04-26T21:08:39.712682+0000","last_fullsized":"2024-04-26T21:08:39.712682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:00.305226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194170+0000","last_change":"2024-04-26T21:08:37.161426+0000","last_active":"2024-04-26T21:08:39.194170+0000","last_peered":"2024-04-26T21:08:39.194170+0000","last_clean":"2024-04-26T21:08:39.194170+0000","last_became_active":"2024-04-26T21:08:37.161119+0000","last_became_peered":"2024-04-26T21:08:37.161119+0000","last_unstale":"2024-04-26T21:08:39.194170+0000","last_undegraded":"2024-04-26T21:08:39.194170+0000","last_fullsized":"2024-04-26T21:08:39.194170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:34.683220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211320+0000","last_change":"2024-04-26T21:08:38.195331+0000","last_active":"2024-04-26T21:08:39.211320+0000","last_peered":"2024-04-26T21:08:39.211320+0000","last_clean":"2024-04-26T21:08:39.211320+0000","last_became_active":"2024-04-26T21:08:38.195002+0000","last_became_peered":"2024-04-26T21:08:38.195002+0000","last_unstale":"2024-04-26T21:08:39.211320+0000","last_undegraded":"2024-04-26T21:08:39.211320+0000","last_fullsized":"2024-04-26T21:08:39.211320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:42.506444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217273+0000","last_change":"2024-04-26T21:08:37.172072+0000","last_active":"2024-04-26T21:08:39.217273+0000","last_peered":"2024-04-26T21:08:39.217273+0000","last_clean":"2024-04-26T21:08:39.217273+0000","last_became_active":"2024-04-26T21:08:37.171778+0000","last_became_peered":"2024-04-26T21:08:37.171778+0000","last_unstale":"2024-04-26T21:08:39.217273+0000","last_undegraded":"2024-04-26T21:08:39.217273+0000","last_fullsized":"2024-04-26T21:08:39.217273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:00.488332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194798+0000","last_change":"2024-04-26T21:08:38.202960+0000","last_active":"2024-04-26T21:08:39.194798+0000","last_peered":"2024-04-26T21:08:39.194798+0000","last_clean":"2024-04-26T21:08:39.194798+0000","last_became_active":"2024-04-26T21:08:38.202642+0000","last_became_peered":"2024-04-26T21:08:38.202642+0000","last_unstale":"2024-04-26T21:08:39.194798+0000","last_undegraded":"2024-04-26T21:08:39.194798+0000","last_fullsized":"2024-04-26T21:08:39.194798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:55.797399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194905+0000","last_change":"2024-04-26T21:08:37.174576+0000","last_active":"2024-04-26T21:08:39.194905+0000","last_peered":"2024-04-26T21:08:39.194905+0000","last_clean":"2024-04-26T21:08:39.194905+0000","last_became_active":"2024-04-26T21:08:37.174223+0000","last_became_peered":"2024-04-26T21:08:37.174223+0000","last_unstale":"2024-04-26T21:08:39.194905+0000","last_undegraded":"2024-04-26T21:08:39.194905+0000","last_fullsized":"2024-04-26T21:08:39.194905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:00.488332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712374+0000","last_change":"2024-04-26T21:08:38.210232+0000","last_active":"2024-04-26T21:08:39.712374+0000","last_peered":"2024-04-26T21:08:39.712374+0000","last_clean":"2024-04-26T21:08:39.712374+0000","last_became_active":"2024-04-26T21:08:38.207957+0000","last_became_peered":"2024-04-26T21:08:38.207957+0000","last_unstale":"2024-04-26T21:08:39.712374+0000","last_undegraded":"2024-04-26T21:08:39.712374+0000","last_fullsized":"2024-04-26T21:08:39.712374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:29.659217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211019+0000","last_change":"2024-04-26T21:08:37.166766+0000","last_active":"2024-04-26T21:08:39.211019+0000","last_peered":"2024-04-26T21:08:39.211019+0000","last_clean":"2024-04-26T21:08:39.211019+0000","last_became_active":"2024-04-26T21:08:37.166441+0000","last_became_peered":"2024-04-26T21:08:37.166441+0000","last_unstale":"2024-04-26T21:08:39.211019+0000","last_undegraded":"2024-04-26T21:08:39.211019+0000","last_fullsized":"2024-04-26T21:08:39.211019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:58.240486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195784+0000","last_change":"2024-04-26T21:08:38.209109+0000","last_active":"2024-04-26T21:08:39.195784+0000","last_peered":"2024-04-26T21:08:39.195784+0000","last_clean":"2024-04-26T21:08:39.195784+0000","last_became_active":"2024-04-26T21:08:38.208779+0000","last_became_peered":"2024-04-26T21:08:38.208779+0000","last_unstale":"2024-04-26T21:08:39.195784+0000","last_undegraded":"2024-04-26T21:08:39.195784+0000","last_fullsized":"2024-04-26T21:08:39.195784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:59.255528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217053+0000","last_change":"2024-04-26T21:08:37.176062+0000","last_active":"2024-04-26T21:08:39.217053+0000","last_peered":"2024-04-26T21:08:39.217053+0000","last_clean":"2024-04-26T21:08:39.217053+0000","last_became_active":"2024-04-26T21:08:37.175757+0000","last_became_peered":"2024-04-26T21:08:37.175757+0000","last_unstale":"2024-04-26T21:08:39.217053+0000","last_undegraded":"2024-04-26T21:08:39.217053+0000","last_fullsized":"2024-04-26T21:08:39.217053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:58.240486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711341+0000","last_change":"2024-04-26T21:08:38.199662+0000","last_active":"2024-04-26T21:08:39.711341+0000","last_peered":"2024-04-26T21:08:39.711341+0000","last_clean":"2024-04-26T21:08:39.711341+0000","last_became_active":"2024-04-26T21:08:38.198909+0000","last_became_peered":"2024-04-26T21:08:38.198909+0000","last_unstale":"2024-04-26T21:08:39.711341+0000","last_undegraded":"2024-04-26T21:08:39.711341+0000","last_fullsized":"2024-04-26T21:08:39.711341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:46.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.713546+0000","last_change":"2024-04-26T21:08:37.149949+0000","last_active":"2024-04-26T21:08:39.713546+0000","last_peered":"2024-04-26T21:08:39.713546+0000","last_clean":"2024-04-26T21:08:39.713546+0000","last_became_active":"2024-04-26T21:08:37.149579+0000","last_became_peered":"2024-04-26T21:08:37.149579+0000","last_unstale":"2024-04-26T21:08:39.713546+0000","last_undegraded":"2024-04-26T21:08:39.713546+0000","last_fullsized":"2024-04-26T21:08:39.713546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:29.045350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712396+0000","last_change":"2024-04-26T21:08:38.196010+0000","last_active":"2024-04-26T21:08:39.712396+0000","last_peered":"2024-04-26T21:08:39.712396+0000","last_clean":"2024-04-26T21:08:39.712396+0000","last_became_active":"2024-04-26T21:08:38.195650+0000","last_became_peered":"2024-04-26T21:08:38.195650+0000","last_unstale":"2024-04-26T21:08:39.712396+0000","last_undegraded":"2024-04-26T21:08:39.712396+0000","last_fullsized":"2024-04-26T21:08:39.712396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:59.807950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194744+0000","last_change":"2024-04-26T21:08:37.170877+0000","last_active":"2024-04-26T21:08:39.194744+0000","last_peered":"2024-04-26T21:08:39.194744+0000","last_clean":"2024-04-26T21:08:39.194744+0000","last_became_active":"2024-04-26T21:08:37.170548+0000","last_became_peered":"2024-04-26T21:08:37.170548+0000","last_unstale":"2024-04-26T21:08:39.194744+0000","last_undegraded":"2024-04-26T21:08:39.194744+0000","last_fullsized":"2024-04-26T21:08:39.194744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:58.240486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711189+0000","last_change":"2024-04-26T21:08:38.217551+0000","last_active":"2024-04-26T21:08:39.711189+0000","last_peered":"2024-04-26T21:08:39.711189+0000","last_clean":"2024-04-26T21:08:39.711189+0000","last_became_active":"2024-04-26T21:08:38.217228+0000","last_became_peered":"2024-04-26T21:08:38.217228+0000","last_unstale":"2024-04-26T21:08:39.711189+0000","last_undegraded":"2024-04-26T21:08:39.711189+0000","last_fullsized":"2024-04-26T21:08:39.711189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:00.305226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.216714+0000","last_change":"2024-04-26T21:08:37.170243+0000","last_active":"2024-04-26T21:08:39.216714+0000","last_peered":"2024-04-26T21:08:39.216714+0000","last_clean":"2024-04-26T21:08:39.216714+0000","last_became_active":"2024-04-26T21:08:37.169995+0000","last_became_peered":"2024-04-26T21:08:37.169995+0000","last_unstale":"2024-04-26T21:08:39.216714+0000","last_undegraded":"2024-04-26T21:08:39.216714+0000","last_fullsized":"2024-04-26T21:08:39.216714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:18.839848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.216725+0000","last_change":"2024-04-26T21:08:38.200818+0000","last_active":"2024-04-26T21:08:39.216725+0000","last_peered":"2024-04-26T21:08:39.216725+0000","last_clean":"2024-04-26T21:08:39.216725+0000","last_became_active":"2024-04-26T21:08:38.198256+0000","last_became_peered":"2024-04-26T21:08:38.198256+0000","last_unstale":"2024-04-26T21:08:39.216725+0000","last_undegraded":"2024-04-26T21:08:39.216725+0000","last_fullsized":"2024-04-26T21:08:39.216725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:01.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.191066+0000","last_change":"2024-04-26T21:08:37.170807+0000","last_active":"2024-04-26T21:08:39.191066+0000","last_peered":"2024-04-26T21:08:39.191066+0000","last_clean":"2024-04-26T21:08:39.191066+0000","last_became_active":"2024-04-26T21:08:37.170428+0000","last_became_peered":"2024-04-26T21:08:37.170428+0000","last_unstale":"2024-04-26T21:08:39.191066+0000","last_undegraded":"2024-04-26T21:08:39.191066+0000","last_fullsized":"2024-04-26T21:08:39.191066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:34.683220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711663+0000","last_change":"2024-04-26T21:08:38.209564+0000","last_active":"2024-04-26T21:08:39.711663+0000","last_peered":"2024-04-26T21:08:39.711663+0000","last_clean":"2024-04-26T21:08:39.711663+0000","last_became_active":"2024-04-26T21:08:38.209194+0000","last_became_peered":"2024-04-26T21:08:38.209194+0000","last_unstale":"2024-04-26T21:08:39.711663+0000","last_undegraded":"2024-04-26T21:08:39.711663+0000","last_fullsized":"2024-04-26T21:08:39.711663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:35.132498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.196061+0000","last_change":"2024-04-26T21:08:37.156599+0000","last_active":"2024-04-26T21:08:39.196061+0000","last_peered":"2024-04-26T21:08:39.196061+0000","last_clean":"2024-04-26T21:08:39.196061+0000","last_became_active":"2024-04-26T21:08:37.154195+0000","last_became_peered":"2024-04-26T21:08:37.154195+0000","last_unstale":"2024-04-26T21:08:39.196061+0000","last_undegraded":"2024-04-26T21:08:39.196061+0000","last_fullsized":"2024-04-26T21:08:39.196061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:00.067702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711404+0000","last_change":"2024-04-26T21:08:38.209445+0000","last_active":"2024-04-26T21:08:39.711404+0000","last_peered":"2024-04-26T21:08:39.711404+0000","last_clean":"2024-04-26T21:08:39.711404+0000","last_became_active":"2024-04-26T21:08:38.209134+0000","last_became_peered":"2024-04-26T21:08:38.209134+0000","last_unstale":"2024-04-26T21:08:39.711404+0000","last_undegraded":"2024-04-26T21:08:39.711404+0000","last_fullsized":"2024-04-26T21:08:39.711404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:44.324217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.191281+0000","last_change":"2024-04-26T21:08:37.176762+0000","last_active":"2024-04-26T21:08:39.191281+0000","last_peered":"2024-04-26T21:08:39.191281+0000","last_clean":"2024-04-26T21:08:39.191281+0000","last_became_active":"2024-04-26T21:08:37.176369+0000","last_became_peered":"2024-04-26T21:08:37.176369+0000","last_unstale":"2024-04-26T21:08:39.191281+0000","last_undegraded":"2024-04-26T21:08:39.191281+0000","last_fullsized":"2024-04-26T21:08:39.191281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:00.488332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712217+0000","last_change":"2024-04-26T21:08:38.211784+0000","last_active":"2024-04-26T21:08:39.712217+0000","last_peered":"2024-04-26T21:08:39.712217+0000","last_clean":"2024-04-26T21:08:39.712217+0000","last_became_active":"2024-04-26T21:08:38.210121+0000","last_became_peered":"2024-04-26T21:08:38.210121+0000","last_unstale":"2024-04-26T21:08:39.712217+0000","last_undegraded":"2024-04-26T21:08:39.712217+0000","last_fullsized":"2024-04-26T21:08:39.712217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:46.194523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211130+0000","last_change":"2024-04-26T21:08:37.167350+0000","last_active":"2024-04-26T21:08:39.211130+0000","last_peered":"2024-04-26T21:08:39.211130+0000","last_clean":"2024-04-26T21:08:39.211130+0000","last_became_active":"2024-04-26T21:08:37.166952+0000","last_became_peered":"2024-04-26T21:08:37.166952+0000","last_unstale":"2024-04-26T21:08:39.211130+0000","last_undegraded":"2024-04-26T21:08:39.211130+0000","last_fullsized":"2024-04-26T21:08:39.211130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:30.628461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195797+0000","last_change":"2024-04-26T21:08:38.202682+0000","last_active":"2024-04-26T21:08:39.195797+0000","last_peered":"2024-04-26T21:08:39.195797+0000","last_clean":"2024-04-26T21:08:39.195797+0000","last_became_active":"2024-04-26T21:08:38.202299+0000","last_became_peered":"2024-04-26T21:08:38.202299+0000","last_unstale":"2024-04-26T21:08:39.195797+0000","last_undegraded":"2024-04-26T21:08:39.195797+0000","last_fullsized":"2024-04-26T21:08:39.195797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:19.756896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.191430+0000","last_change":"2024-04-26T21:08:37.168687+0000","last_active":"2024-04-26T21:08:39.191430+0000","last_peered":"2024-04-26T21:08:39.191430+0000","last_clean":"2024-04-26T21:08:39.191430+0000","last_became_active":"2024-04-26T21:08:37.168302+0000","last_became_peered":"2024-04-26T21:08:37.168302+0000","last_unstale":"2024-04-26T21:08:39.191430+0000","last_undegraded":"2024-04-26T21:08:39.191430+0000","last_fullsized":"2024-04-26T21:08:39.191430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:48:02.545851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711826+0000","last_change":"2024-04-26T21:08:38.208610+0000","last_active":"2024-04-26T21:08:39.711826+0000","last_peered":"2024-04-26T21:08:39.711826+0000","last_clean":"2024-04-26T21:08:39.711826+0000","last_became_active":"2024-04-26T21:08:38.206735+0000","last_became_peered":"2024-04-26T21:08:38.206735+0000","last_unstale":"2024-04-26T21:08:39.711826+0000","last_undegraded":"2024-04-26T21:08:39.711826+0000","last_fullsized":"2024-04-26T21:08:39.711826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:54.952928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.710746+0000","last_change":"2024-04-26T21:08:33.504911+0000","last_active":"2024-04-26T21:08:39.710746+0000","last_peered":"2024-04-26T21:08:39.710746+0000","last_clean":"2024-04-26T21:08:39.710746+0000","last_became_active":"2024-04-26T21:08:33.503667+0000","last_became_peered":"2024-04-26T21:08:33.503667+0000","last_unstale":"2024-04-26T21:08:39.710746+0000","last_undegraded":"2024-04-26T21:08:39.710746+0000","last_fullsized":"2024-04-26T21:08:39.710746+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:32.013907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:32.013907+0000","last_clean_scrub_stamp":"2024-04-26T21:08:32.013907+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:09.596153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194166+0000","last_change":"2024-04-26T21:08:37.161497+0000","last_active":"2024-04-26T21:08:39.194166+0000","last_peered":"2024-04-26T21:08:39.194166+0000","last_clean":"2024-04-26T21:08:39.194166+0000","last_became_active":"2024-04-26T21:08:37.161144+0000","last_became_peered":"2024-04-26T21:08:37.161144+0000","last_unstale":"2024-04-26T21:08:39.194166+0000","last_undegraded":"2024-04-26T21:08:39.194166+0000","last_fullsized":"2024-04-26T21:08:39.194166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:18.839848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712097+0000","last_change":"2024-04-26T21:08:38.211227+0000","last_active":"2024-04-26T21:08:39.712097+0000","last_peered":"2024-04-26T21:08:39.712097+0000","last_clean":"2024-04-26T21:08:39.712097+0000","last_became_active":"2024-04-26T21:08:38.210882+0000","last_became_peered":"2024-04-26T21:08:38.210882+0000","last_unstale":"2024-04-26T21:08:39.712097+0000","last_undegraded":"2024-04-26T21:08:39.712097+0000","last_fullsized":"2024-04-26T21:08:39.712097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:42.506444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.710561+0000","last_change":"2024-04-26T21:08:37.177216+0000","last_active":"2024-04-26T21:08:39.710561+0000","last_peered":"2024-04-26T21:08:39.710561+0000","last_clean":"2024-04-26T21:08:39.710561+0000","last_became_active":"2024-04-26T21:08:37.176919+0000","last_became_peered":"2024-04-26T21:08:37.176919+0000","last_unstale":"2024-04-26T21:08:39.710561+0000","last_undegraded":"2024-04-26T21:08:39.710561+0000","last_fullsized":"2024-04-26T21:08:39.710561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:58.240486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712236+0000","last_change":"2024-04-26T21:08:38.191994+0000","last_active":"2024-04-26T21:08:39.712236+0000","last_peered":"2024-04-26T21:08:39.712236+0000","last_clean":"2024-04-26T21:08:39.712236+0000","last_became_active":"2024-04-26T21:08:38.191567+0000","last_became_peered":"2024-04-26T21:08:38.191567+0000","last_unstale":"2024-04-26T21:08:39.712236+0000","last_undegraded":"2024-04-26T21:08:39.712236+0000","last_fullsized":"2024-04-26T21:08:39.712236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:01.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.191022+0000","last_change":"2024-04-26T21:08:37.172777+0000","last_active":"2024-04-26T21:08:39.191022+0000","last_peered":"2024-04-26T21:08:39.191022+0000","last_clean":"2024-04-26T21:08:39.191022+0000","last_became_active":"2024-04-26T21:08:37.171865+0000","last_became_peered":"2024-04-26T21:08:37.171865+0000","last_unstale":"2024-04-26T21:08:39.191022+0000","last_undegraded":"2024-04-26T21:08:39.191022+0000","last_fullsized":"2024-04-26T21:08:39.191022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:18.839848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.210976+0000","last_change":"2024-04-26T21:08:38.200547+0000","last_active":"2024-04-26T21:08:39.210976+0000","last_peered":"2024-04-26T21:08:39.210976+0000","last_clean":"2024-04-26T21:08:39.210976+0000","last_became_active":"2024-04-26T21:08:38.200025+0000","last_became_peered":"2024-04-26T21:08:38.200025+0000","last_unstale":"2024-04-26T21:08:39.210976+0000","last_undegraded":"2024-04-26T21:08:39.210976+0000","last_fullsized":"2024-04-26T21:08:39.210976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:59.807950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194881+0000","last_change":"2024-04-26T21:08:37.170761+0000","last_active":"2024-04-26T21:08:39.194881+0000","last_peered":"2024-04-26T21:08:39.194881+0000","last_clean":"2024-04-26T21:08:39.194881+0000","last_became_active":"2024-04-26T21:08:37.170431+0000","last_became_peered":"2024-04-26T21:08:37.170431+0000","last_unstale":"2024-04-26T21:08:39.194881+0000","last_undegraded":"2024-04-26T21:08:39.194881+0000","last_fullsized":"2024-04-26T21:08:39.194881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:59.977930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195387+0000","last_change":"2024-04-26T21:08:38.213352+0000","last_active":"2024-04-26T21:08:39.195387+0000","last_peered":"2024-04-26T21:08:39.195387+0000","last_clean":"2024-04-26T21:08:39.195387+0000","last_became_active":"2024-04-26T21:08:38.201057+0000","last_became_peered":"2024-04-26T21:08:38.201057+0000","last_unstale":"2024-04-26T21:08:39.195387+0000","last_undegraded":"2024-04-26T21:08:39.195387+0000","last_fullsized":"2024-04-26T21:08:39.195387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:35.698263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.210868+0000","last_change":"2024-04-26T21:08:37.169249+0000","last_active":"2024-04-26T21:08:39.210868+0000","last_peered":"2024-04-26T21:08:39.210868+0000","last_clean":"2024-04-26T21:08:39.210868+0000","last_became_active":"2024-04-26T21:08:37.168910+0000","last_became_peered":"2024-04-26T21:08:37.168910+0000","last_unstale":"2024-04-26T21:08:39.210868+0000","last_undegraded":"2024-04-26T21:08:39.210868+0000","last_fullsized":"2024-04-26T21:08:39.210868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:34.683220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194515+0000","last_change":"2024-04-26T21:08:38.206722+0000","last_active":"2024-04-26T21:08:39.194515+0000","last_peered":"2024-04-26T21:08:39.194515+0000","last_clean":"2024-04-26T21:08:39.194515+0000","last_became_active":"2024-04-26T21:08:38.205394+0000","last_became_peered":"2024-04-26T21:08:38.205394+0000","last_unstale":"2024-04-26T21:08:39.194515+0000","last_undegraded":"2024-04-26T21:08:39.194515+0000","last_fullsized":"2024-04-26T21:08:39.194515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:44.324217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712111+0000","last_change":"2024-04-26T21:08:37.178417+0000","last_active":"2024-04-26T21:08:39.712111+0000","last_peered":"2024-04-26T21:08:39.712111+0000","last_clean":"2024-04-26T21:08:39.712111+0000","last_became_active":"2024-04-26T21:08:37.178031+0000","last_became_peered":"2024-04-26T21:08:37.178031+0000","last_unstale":"2024-04-26T21:08:39.712111+0000","last_undegraded":"2024-04-26T21:08:39.712111+0000","last_fullsized":"2024-04-26T21:08:39.712111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:54.872853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.210824+0000","last_change":"2024-04-26T21:08:38.200358+0000","last_active":"2024-04-26T21:08:39.210824+0000","last_peered":"2024-04-26T21:08:39.210824+0000","last_clean":"2024-04-26T21:08:39.210824+0000","last_became_active":"2024-04-26T21:08:38.200001+0000","last_became_peered":"2024-04-26T21:08:38.200001+0000","last_unstale":"2024-04-26T21:08:39.210824+0000","last_undegraded":"2024-04-26T21:08:39.210824+0000","last_fullsized":"2024-04-26T21:08:39.210824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:01.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.215005+0000","last_change":"2024-04-26T21:08:37.178473+0000","last_active":"2024-04-26T21:08:39.215005+0000","last_peered":"2024-04-26T21:08:39.215005+0000","last_clean":"2024-04-26T21:08:39.215005+0000","last_became_active":"2024-04-26T21:08:37.178079+0000","last_became_peered":"2024-04-26T21:08:37.178079+0000","last_unstale":"2024-04-26T21:08:39.215005+0000","last_undegraded":"2024-04-26T21:08:39.215005+0000","last_fullsized":"2024-04-26T21:08:39.215005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:31.424853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712508+0000","last_change":"2024-04-26T21:08:38.212830+0000","last_active":"2024-04-26T21:08:39.712508+0000","last_peered":"2024-04-26T21:08:39.712508+0000","last_clean":"2024-04-26T21:08:39.712508+0000","last_became_active":"2024-04-26T21:08:38.212327+0000","last_became_peered":"2024-04-26T21:08:38.212327+0000","last_unstale":"2024-04-26T21:08:39.712508+0000","last_undegraded":"2024-04-26T21:08:39.712508+0000","last_fullsized":"2024-04-26T21:08:39.712508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:26.983477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217434+0000","last_change":"2024-04-26T21:08:37.172503+0000","last_active":"2024-04-26T21:08:39.217434+0000","last_peered":"2024-04-26T21:08:39.217434+0000","last_clean":"2024-04-26T21:08:39.217434+0000","last_became_active":"2024-04-26T21:08:37.172129+0000","last_became_peered":"2024-04-26T21:08:37.172129+0000","last_unstale":"2024-04-26T21:08:39.217434+0000","last_undegraded":"2024-04-26T21:08:39.217434+0000","last_fullsized":"2024-04-26T21:08:39.217434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:34.683220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712025+0000","last_change":"2024-04-26T21:08:38.192302+0000","last_active":"2024-04-26T21:08:39.712025+0000","last_peered":"2024-04-26T21:08:39.712025+0000","last_clean":"2024-04-26T21:08:39.712025+0000","last_became_active":"2024-04-26T21:08:38.191741+0000","last_became_peered":"2024-04-26T21:08:38.191741+0000","last_unstale":"2024-04-26T21:08:39.712025+0000","last_undegraded":"2024-04-26T21:08:39.712025+0000","last_fullsized":"2024-04-26T21:08:39.712025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:48.443740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.710597+0000","last_change":"2024-04-26T21:08:37.178123+0000","last_active":"2024-04-26T21:08:39.710597+0000","last_peered":"2024-04-26T21:08:39.710597+0000","last_clean":"2024-04-26T21:08:39.710597+0000","last_became_active":"2024-04-26T21:08:37.177831+0000","last_became_peered":"2024-04-26T21:08:37.177831+0000","last_unstale":"2024-04-26T21:08:39.710597+0000","last_undegraded":"2024-04-26T21:08:39.710597+0000","last_fullsized":"2024-04-26T21:08:39.710597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:59.977930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.196679+0000","last_change":"2024-04-26T21:08:38.212239+0000","last_active":"2024-04-26T21:08:39.196679+0000","last_peered":"2024-04-26T21:08:39.196679+0000","last_clean":"2024-04-26T21:08:39.196679+0000","last_became_active":"2024-04-26T21:08:38.209717+0000","last_became_peered":"2024-04-26T21:08:38.209717+0000","last_unstale":"2024-04-26T21:08:39.196679+0000","last_undegraded":"2024-04-26T21:08:39.196679+0000","last_fullsized":"2024-04-26T21:08:39.196679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:31.643503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211322+0000","last_change":"2024-04-26T21:08:37.173887+0000","last_active":"2024-04-26T21:08:39.211322+0000","last_peered":"2024-04-26T21:08:39.211322+0000","last_clean":"2024-04-26T21:08:39.211322+0000","last_became_active":"2024-04-26T21:08:37.173034+0000","last_became_peered":"2024-04-26T21:08:37.173034+0000","last_unstale":"2024-04-26T21:08:39.211322+0000","last_undegraded":"2024-04-26T21:08:39.211322+0000","last_fullsized":"2024-04-26T21:08:39.211322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:00.488332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712910+0000","last_change":"2024-04-26T21:08:38.197901+0000","last_active":"2024-04-26T21:08:39.712910+0000","last_peered":"2024-04-26T21:08:39.712910+0000","last_clean":"2024-04-26T21:08:39.712910+0000","last_became_active":"2024-04-26T21:08:38.197592+0000","last_became_peered":"2024-04-26T21:08:38.197592+0000","last_unstale":"2024-04-26T21:08:39.712910+0000","last_undegraded":"2024-04-26T21:08:39.712910+0000","last_fullsized":"2024-04-26T21:08:39.712910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:46.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211038+0000","last_change":"2024-04-26T21:08:37.175903+0000","last_active":"2024-04-26T21:08:39.211038+0000","last_peered":"2024-04-26T21:08:39.211038+0000","last_clean":"2024-04-26T21:08:39.211038+0000","last_became_active":"2024-04-26T21:08:37.175605+0000","last_became_peered":"2024-04-26T21:08:37.175605+0000","last_unstale":"2024-04-26T21:08:39.211038+0000","last_undegraded":"2024-04-26T21:08:39.211038+0000","last_fullsized":"2024-04-26T21:08:39.211038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:59.977930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711599+0000","last_change":"2024-04-26T21:08:38.208922+0000","last_active":"2024-04-26T21:08:39.711599+0000","last_peered":"2024-04-26T21:08:39.711599+0000","last_clean":"2024-04-26T21:08:39.711599+0000","last_became_active":"2024-04-26T21:08:38.208597+0000","last_became_peered":"2024-04-26T21:08:38.208597+0000","last_unstale":"2024-04-26T21:08:39.711599+0000","last_undegraded":"2024-04-26T21:08:39.711599+0000","last_fullsized":"2024-04-26T21:08:39.711599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:59.807950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711492+0000","last_change":"2024-04-26T21:08:37.173916+0000","last_active":"2024-04-26T21:08:39.711492+0000","last_peered":"2024-04-26T21:08:39.711492+0000","last_clean":"2024-04-26T21:08:39.711492+0000","last_became_active":"2024-04-26T21:08:37.173553+0000","last_became_peered":"2024-04-26T21:08:37.173553+0000","last_unstale":"2024-04-26T21:08:39.711492+0000","last_undegraded":"2024-04-26T21:08:39.711492+0000","last_fullsized":"2024-04-26T21:08:39.711492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:30.628461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217095+0000","last_change":"2024-04-26T21:08:38.212699+0000","last_active":"2024-04-26T21:08:39.217095+0000","last_peered":"2024-04-26T21:08:39.217095+0000","last_clean":"2024-04-26T21:08:39.217095+0000","last_became_active":"2024-04-26T21:08:38.212418+0000","last_became_peered":"2024-04-26T21:08:38.212418+0000","last_unstale":"2024-04-26T21:08:39.217095+0000","last_undegraded":"2024-04-26T21:08:39.217095+0000","last_fullsized":"2024-04-26T21:08:39.217095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:59.807950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194366+0000","last_change":"2024-04-26T21:08:38.211036+0000","last_active":"2024-04-26T21:08:39.194366+0000","last_peered":"2024-04-26T21:08:39.194366+0000","last_clean":"2024-04-26T21:08:39.194366+0000","last_became_active":"2024-04-26T21:08:38.207129+0000","last_became_peered":"2024-04-26T21:08:38.207129+0000","last_unstale":"2024-04-26T21:08:39.194366+0000","last_undegraded":"2024-04-26T21:08:39.194366+0000","last_fullsized":"2024-04-26T21:08:39.194366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:46.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711252+0000","last_change":"2024-04-26T21:08:37.179811+0000","last_active":"2024-04-26T21:08:39.711252+0000","last_peered":"2024-04-26T21:08:39.711252+0000","last_clean":"2024-04-26T21:08:39.711252+0000","last_became_active":"2024-04-26T21:08:37.179425+0000","last_became_peered":"2024-04-26T21:08:37.179425+0000","last_unstale":"2024-04-26T21:08:39.711252+0000","last_undegraded":"2024-04-26T21:08:39.711252+0000","last_fullsized":"2024-04-26T21:08:39.711252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:31.424853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195669+0000","last_change":"2024-04-26T21:08:38.207278+0000","last_active":"2024-04-26T21:08:39.195669+0000","last_peered":"2024-04-26T21:08:39.195669+0000","last_clean":"2024-04-26T21:08:39.195669+0000","last_became_active":"2024-04-26T21:08:38.204849+0000","last_became_peered":"2024-04-26T21:08:38.204849+0000","last_unstale":"2024-04-26T21:08:39.195669+0000","last_undegraded":"2024-04-26T21:08:39.195669+0000","last_fullsized":"2024-04-26T21:08:39.195669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:54.952928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711653+0000","last_change":"2024-04-26T21:08:37.175992+0000","last_active":"2024-04-26T21:08:39.711653+0000","last_peered":"2024-04-26T21:08:39.711653+0000","last_clean":"2024-04-26T21:08:39.711653+0000","last_became_active":"2024-04-26T21:08:37.175682+0000","last_became_peered":"2024-04-26T21:08:37.175682+0000","last_unstale":"2024-04-26T21:08:39.711653+0000","last_undegraded":"2024-04-26T21:08:39.711653+0000","last_fullsized":"2024-04-26T21:08:39.711653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:33.374984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.710839+0000","last_change":"2024-04-26T21:08:38.207957+0000","last_active":"2024-04-26T21:08:39.710839+0000","last_peered":"2024-04-26T21:08:39.710839+0000","last_clean":"2024-04-26T21:08:39.710839+0000","last_became_active":"2024-04-26T21:08:38.207573+0000","last_became_peered":"2024-04-26T21:08:38.207573+0000","last_unstale":"2024-04-26T21:08:39.710839+0000","last_undegraded":"2024-04-26T21:08:39.710839+0000","last_fullsized":"2024-04-26T21:08:39.710839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:02:14.307645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712093+0000","last_change":"2024-04-26T21:08:37.173718+0000","last_active":"2024-04-26T21:08:39.712093+0000","last_peered":"2024-04-26T21:08:39.712093+0000","last_clean":"2024-04-26T21:08:39.712093+0000","last_became_active":"2024-04-26T21:08:37.170846+0000","last_became_peered":"2024-04-26T21:08:37.170846+0000","last_unstale":"2024-04-26T21:08:39.712093+0000","last_undegraded":"2024-04-26T21:08:39.712093+0000","last_fullsized":"2024-04-26T21:08:39.712093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:48:02.545851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711803+0000","last_change":"2024-04-26T21:08:38.199029+0000","last_active":"2024-04-26T21:08:39.711803+0000","last_peered":"2024-04-26T21:08:39.711803+0000","last_clean":"2024-04-26T21:08:39.711803+0000","last_became_active":"2024-04-26T21:08:38.198720+0000","last_became_peered":"2024-04-26T21:08:38.198720+0000","last_unstale":"2024-04-26T21:08:39.711803+0000","last_undegraded":"2024-04-26T21:08:39.711803+0000","last_fullsized":"2024-04-26T21:08:39.711803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:40.618646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712817+0000","last_change":"2024-04-26T21:08:37.165057+0000","last_active":"2024-04-26T21:08:39.712817+0000","last_peered":"2024-04-26T21:08:39.712817+0000","last_clean":"2024-04-26T21:08:39.712817+0000","last_became_active":"2024-04-26T21:08:37.164781+0000","last_became_peered":"2024-04-26T21:08:37.164781+0000","last_unstale":"2024-04-26T21:08:39.712817+0000","last_undegraded":"2024-04-26T21:08:39.712817+0000","last_fullsized":"2024-04-26T21:08:39.712817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:17.499100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:08:43.128 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:08:43.129 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T21:08:43.129 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-26T21:08:43.129 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-26T21:08:43.129 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-26T21:08:43.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.387+0000 7fefa2600640 1 Processor -- start 2024-04-26T21:08:43.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.387+0000 7fefa2600640 1 -- start start 2024-04-26T21:08:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.389+0000 7fefa2600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fef9c0d06f0 0x7fef9c0d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.389+0000 7fefa2600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fef9c0d1670 0x7fef9c071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.389+0000 7fefa2600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef9c0cf690 con 0x7fef9c145c20 2024-04-26T21:08:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.389+0000 7fefa2600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fef9c0ced10 con 0x7fef9c0d1670 2024-04-26T21:08:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.389+0000 7fefa2600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fef9c0d1ac0 con 0x7fef9c0d06f0 2024-04-26T21:08:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.389+0000 7fefa1200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fef9c145c20 0x7fef9c146450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44706/0 (socket says 172.21.15.26:44706) 2024-04-26T21:08:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.389+0000 7fefa1200640 1 -- 172.21.15.26:0/2553310802 learned_addr learned my addr 172.21.15.26:0/2553310802 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.389+0000 7fefa1c00640 1 --2- 172.21.15.26:0/2553310802 >> v2:172.21.15.26:3300/0 conn(0x7fef9c0d1670 0x7fef9c071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:43.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.390+0000 7fef93e00640 1 --2- 172.21.15.26:0/2553310802 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fef9c0d06f0 0x7fef9c0d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:43.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.390+0000 7fefa1c00640 1 -- 172.21.15.26:0/2553310802 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fef9c0d06f0 msgr2=0x7fef9c0d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:43.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.390+0000 7fefa1c00640 1 --2- 172.21.15.26:0/2553310802 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fef9c0d06f0 0x7fef9c0d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:43.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.390+0000 7fefa1c00640 1 -- 172.21.15.26:0/2553310802 >> v1:172.21.15.26:6789/0 conn(0x7fef9c145c20 legacy=0x7fef9c146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:43.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.390+0000 7fefa1c00640 1 -- 172.21.15.26:0/2553310802 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9c0cf930 con 0x7fef9c0d1670 2024-04-26T21:08:43.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.390+0000 7fef9be00640 1 -- 172.21.15.26:0/2553310802 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2371387630 0 0) 0x7fef9c0cf690 con 0x7fef9c145c20 2024-04-26T21:08:43.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.391+0000 7fefa1c00640 1 --2- 172.21.15.26:0/2553310802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef9c0d1670 0x7fef9c071fe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fef940097a0 tx=0x7fef9402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f84154413281c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:43.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.391+0000 7fef9be00640 1 -- 172.21.15.26:0/2553310802 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef94037070 con 0x7fef9c0d1670 2024-04-26T21:08:43.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.391+0000 7fef9be00640 1 -- 172.21.15.26:0/2553310802 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef94032440 con 0x7fef9c0d1670 2024-04-26T21:08:43.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.391+0000 7fef9be00640 1 -- 172.21.15.26:0/2553310802 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef94032810 con 0x7fef9c0d1670 2024-04-26T21:08:43.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.392+0000 7fefa2600640 1 -- 172.21.15.26:0/2553310802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef9c0d1670 msgr2=0x7fef9c071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:43.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.392+0000 7fefa2600640 1 --2- 172.21.15.26:0/2553310802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef9c0d1670 0x7fef9c071fe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fef940097a0 tx=0x7fef9402a180 comp rx=0 tx=0).stop 2024-04-26T21:08:43.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.392+0000 7fefa2600640 1 -- 172.21.15.26:0/2553310802 shutdown_connections 2024-04-26T21:08:43.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.392+0000 7fefa2600640 1 --2- 172.21.15.26:0/2553310802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef9c0d1670 0x7fef9c071fe0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:43.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.392+0000 7fefa2600640 1 --2- 172.21.15.26:0/2553310802 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fef9c0d06f0 0x7fef9c0d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:43.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.392+0000 7fefa2600640 1 -- 172.21.15.26:0/2553310802 >> 172.21.15.26:0/2553310802 conn(0x7fef9c06c690 msgr2=0x7fef9c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:43.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.392+0000 7fefa2600640 1 -- 172.21.15.26:0/2553310802 shutdown_connections 2024-04-26T21:08:43.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.392+0000 7fefa2600640 1 -- 172.21.15.26:0/2553310802 wait complete. 2024-04-26T21:08:43.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.394+0000 7fefa2600640 1 Processor -- start 2024-04-26T21:08:43.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.394+0000 7fefa2600640 1 -- start start 2024-04-26T21:08:43.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.395+0000 7fefa2600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fef9c0d06f0 0x7fef9c141450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:43.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.395+0000 7fefa2600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef9c0d1670 0x7fef9c15da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:43.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.395+0000 7fefa2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef9c145c20 0x7fef9c15df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:43.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.395+0000 7fefa2600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c0cfef0 con 0x7fef9c0d1670 2024-04-26T21:08:43.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.395+0000 7fefa2600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c0d0b50 con 0x7fef9c145c20 2024-04-26T21:08:43.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.395+0000 7fefa2600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fef9c0d01d0 con 0x7fef9c0d06f0 2024-04-26T21:08:43.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.395+0000 7fef93e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef9c0d1670 0x7fef9c15da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:43.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.395+0000 7fef93e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef9c0d1670 0x7fef9c15da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34488/0 (socket says 172.21.15.26:34488) 2024-04-26T21:08:43.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.395+0000 7fef93e00640 1 -- 172.21.15.26:0/2232495597 learned_addr learned my addr 172.21.15.26:0/2232495597 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:43.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.395+0000 7fefa1200640 1 --2- 172.21.15.26:0/2232495597 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fef9c0d06f0 0x7fef9c141450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:43.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.396+0000 7fef93e00640 1 -- 172.21.15.26:0/2232495597 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fef9c0d06f0 msgr2=0x7fef9c141450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:43.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.396+0000 7fef93e00640 1 --2- 172.21.15.26:0/2232495597 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fef9c0d06f0 0x7fef9c141450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:43.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.396+0000 7fef93e00640 1 -- 172.21.15.26:0/2232495597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef9c145c20 msgr2=0x7fef9c15df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:43.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.396+0000 7fef93e00640 1 --2- 172.21.15.26:0/2232495597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef9c145c20 0x7fef9c15df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:43.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.396+0000 7fef93e00640 1 -- 172.21.15.26:0/2232495597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9c0b3e80 con 0x7fef9c0d1670 2024-04-26T21:08:43.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.396+0000 7fef93e00640 1 --2- 172.21.15.26:0/2232495597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef9c0d1670 0x7fef9c15da50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fef8400c820 tx=0x7fef8400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:43.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.396+0000 7fef99600640 1 -- 172.21.15.26:0/2232495597 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84015070 con 0x7fef9c0d1670 2024-04-26T21:08:43.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.396+0000 7fef99600640 1 -- 172.21.15.26:0/2232495597 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef84004240 con 0x7fef9c0d1670 2024-04-26T21:08:43.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.396+0000 7fef99600640 1 -- 172.21.15.26:0/2232495597 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8400ce10 con 0x7fef9c0d1670 2024-04-26T21:08:43.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.396+0000 7fefa2600640 1 -- 172.21.15.26:0/2232495597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9c133410 con 0x7fef9c0d1670 2024-04-26T21:08:43.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.396+0000 7fefa2600640 1 -- 172.21.15.26:0/2232495597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef9c130af0 con 0x7fef9c0d1670 2024-04-26T21:08:43.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.397+0000 7fefa2600640 1 -- 172.21.15.26:0/2232495597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef9c06a700 con 0x7fef9c0d1670 2024-04-26T21:08:43.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.398+0000 7fef99600640 1 -- 172.21.15.26:0/2232495597 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef84007830 con 0x7fef9c0d1670 2024-04-26T21:08:43.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.399+0000 7fef99600640 1 --2- 172.21.15.26:0/2232495597 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fef7806c920 0x7fef7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:43.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.399+0000 7fef99600640 1 -- 172.21.15.26:0/2232495597 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fef8408ba20 con 0x7fef9c0d1670 2024-04-26T21:08:43.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.399+0000 7fefa1200640 1 --2- 172.21.15.26:0/2232495597 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fef7806c920 0x7fef7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:43.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.400+0000 7fefa1200640 1 --2- 172.21.15.26:0/2232495597 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fef7806c920 0x7fef7806ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0012e0 tx=0x7fef8c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:43.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.404+0000 7fef99600640 1 -- 172.21.15.26:0/2232495597 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fef9c0b3e80 con 0x7fef9c0d1670 2024-04-26T21:08:43.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.649+0000 7fefa2600640 1 -- 172.21.15.26:0/2232495597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fef9c141990 con 0x7fef9c0d1670 2024-04-26T21:08:43.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.655+0000 7fef99600640 1 -- 172.21.15.26:0/2232495597 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174135 (secure 0 0 0) 0x7fef84059e40 con 0x7fef9c0d1670 2024-04-26T21:08:43.658 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.658+0000 7fefa2600640 1 -- 172.21.15.26:0/2232495597 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fef7806c920 msgr2=0x7fef7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.658+0000 7fefa2600640 1 --2- 172.21.15.26:0/2232495597 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fef7806c920 0x7fef7806ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0012e0 tx=0x7fef8c009040 comp rx=0 tx=0).stop 2024-04-26T21:08:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.658+0000 7fefa2600640 1 -- 172.21.15.26:0/2232495597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef9c0d1670 msgr2=0x7fef9c15da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.658+0000 7fefa2600640 1 --2- 172.21.15.26:0/2232495597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef9c0d1670 0x7fef9c15da50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fef8400c820 tx=0x7fef8400cce0 comp rx=0 tx=0).stop 2024-04-26T21:08:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.658+0000 7fefa2600640 1 -- 172.21.15.26:0/2232495597 shutdown_connections 2024-04-26T21:08:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.658+0000 7fefa2600640 1 --2- 172.21.15.26:0/2232495597 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fef7806c920 0x7fef7806ede0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.659+0000 7fefa2600640 1 --2- 172.21.15.26:0/2232495597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef9c145c20 0x7fef9c15df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.659+0000 7fefa2600640 1 --2- 172.21.15.26:0/2232495597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef9c0d1670 0x7fef9c15da50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.659+0000 7fefa2600640 1 --2- 172.21.15.26:0/2232495597 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fef9c0d06f0 0x7fef9c141450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.659+0000 7fefa2600640 1 -- 172.21.15.26:0/2232495597 >> 172.21.15.26:0/2232495597 conn(0x7fef9c06c690 msgr2=0x7fef9c15fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.659+0000 7fefa2600640 1 -- 172.21.15.26:0/2232495597 shutdown_connections 2024-04-26T21:08:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.659+0000 7fefa2600640 1 -- 172.21.15.26:0/2232495597 wait complete. 2024-04-26T21:08:43.676 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3130430358},{"type":"v1","addr":"172.21.15.26:6833","nonce":3130430358}]},"active_addr":"172.21.15.26:6833/3130430358","active_change":"2024-04-26T21:08:23.261632+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4111,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":132730251}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":3042817555}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":710283970}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":1555926394}]}]} 2024-04-26T21:08:43.680 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-26T21:08:43.680 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-26T21:08:43.680 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:08:43.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.885+0000 7f544f200640 1 Processor -- start 2024-04-26T21:08:43.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.886+0000 7f544f200640 1 -- start start 2024-04-26T21:08:43.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.887+0000 7f544f200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5450162570 0x7f5450162980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:43.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.887+0000 7f544f200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54500d3d80 con 0x7f54500d1980 2024-04-26T21:08:43.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.887+0000 7f544f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54500cfe90 con 0x7f54500d1d60 2024-04-26T21:08:43.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.887+0000 7f544f200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f54500d04e0 con 0x7f5450162570 2024-04-26T21:08:43.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.887+0000 7f544de00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f54500d1980 0x7f54500d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44716/0 (socket says 172.21.15.26:44716) 2024-04-26T21:08:43.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.887+0000 7f544de00640 1 -- 172.21.15.26:0/3989590904 learned_addr learned my addr 172.21.15.26:0/3989590904 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:43.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.888+0000 7f544e800640 1 --2- 172.21.15.26:0/3989590904 >> v2:172.21.15.86:3300/0 conn(0x7f5450162570 0x7f5450162980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:43.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.888+0000 7f544d400640 1 -- 172.21.15.26:0/3989590904 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1047876639 0 0) 0x7f54500d3d80 con 0x7f54500d1980 2024-04-26T21:08:43.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.888+0000 7f544d400640 1 -- 172.21.15.26:0/3989590904 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5434003520 con 0x7f54500d1980 2024-04-26T21:08:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.888+0000 7f544d400640 1 -- 172.21.15.26:0/3989590904 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3494580721 0 0) 0x7f5434003520 con 0x7f54500d1980 2024-04-26T21:08:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.888+0000 7f544d400640 1 -- 172.21.15.26:0/3989590904 >> v2:172.21.15.86:3300/0 conn(0x7f5450162570 msgr2=0x7f5450162980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.888+0000 7f544d400640 1 --2- 172.21.15.26:0/3989590904 >> v2:172.21.15.86:3300/0 conn(0x7f5450162570 0x7f5450162980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.888+0000 7f544d400640 1 -- 172.21.15.26:0/3989590904 >> v1:172.21.15.86:6789/0 conn(0x7f54500d1d60 legacy=0x7f54500b7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.888+0000 7f544d400640 1 -- 172.21.15.26:0/3989590904 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54500d0170 con 0x7f54500d1980 2024-04-26T21:08:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.888+0000 7f544d400640 1 -- 172.21.15.26:0/3989590904 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f54400019f0 con 0x7f54500d1980 2024-04-26T21:08:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.888+0000 7f544d400640 1 -- 172.21.15.26:0/3989590904 >> v1:172.21.15.26:6789/0 conn(0x7f54500d1980 legacy=0x7f54500d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.888+0000 7f544d400640 1 --2- 172.21.15.26:0/3989590904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54340039b0 0x7f5434003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.888+0000 7f544d400640 1 --2- 172.21.15.26:0/3989590904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5434004800 0x7f5434004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.888+0000 7f544d400640 1 --2- 172.21.15.26:0/3989590904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f54340051c0 0x7f543400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.889+0000 7f544d400640 1 -- 172.21.15.26:0/3989590904 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f54500d04e0 con 0x7f5434004800 2024-04-26T21:08:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.889+0000 7f544d400640 1 -- 172.21.15.26:0/3989590904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f543400c060 con 0x7f54340039b0 2024-04-26T21:08:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.889+0000 7f544d400640 1 -- 172.21.15.26:0/3989590904 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f543400c260 con 0x7f54340051c0 2024-04-26T21:08:43.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.889+0000 7f5445400640 1 --2- 172.21.15.26:0/3989590904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5434004800 0x7f5434004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:43.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.889+0000 7f544de00640 1 --2- 172.21.15.26:0/3989590904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54340039b0 0x7f5434003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:43.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.889+0000 7f544e800640 1 --2- 172.21.15.26:0/3989590904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f54340051c0 0x7f543400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:43.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.889+0000 7f5445400640 1 -- 172.21.15.26:0/3989590904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f54340051c0 msgr2=0x7f543400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:43.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.889+0000 7f5445400640 1 --2- 172.21.15.26:0/3989590904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f54340051c0 0x7f543400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:43.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.889+0000 7f5445400640 1 -- 172.21.15.26:0/3989590904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54340039b0 msgr2=0x7f5434003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:43.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.889+0000 7f5445400640 1 --2- 172.21.15.26:0/3989590904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54340039b0 0x7f5434003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:43.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.889+0000 7f5445400640 1 -- 172.21.15.26:0/3989590904 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5434003520 con 0x7f5434004800 2024-04-26T21:08:43.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.889+0000 7f544e800640 1 -- 172.21.15.26:0/3989590904 reap_dead start 2024-04-26T21:08:43.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.890+0000 7f5445400640 1 --2- 172.21.15.26:0/3989590904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5434004800 0x7f5434004c80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5438001750 tx=0x7f5434005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f87b7314ffb13de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:43.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.890+0000 7f544d400640 1 -- 172.21.15.26:0/3989590904 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438001370 con 0x7f5434004800 2024-04-26T21:08:43.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.890+0000 7f544d400640 1 -- 172.21.15.26:0/3989590904 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54380052d0 con 0x7f5434004800 2024-04-26T21:08:43.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.890+0000 7f544d400640 1 -- 172.21.15.26:0/3989590904 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543802abc0 con 0x7f5434004800 2024-04-26T21:08:43.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.891+0000 7f544f200640 1 -- 172.21.15.26:0/3989590904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5434004800 msgr2=0x7f5434004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:43.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.891+0000 7f544f200640 1 --2- 172.21.15.26:0/3989590904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5434004800 0x7f5434004c80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5438001750 tx=0x7f5434005830 comp rx=0 tx=0).stop 2024-04-26T21:08:43.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.891+0000 7f544f200640 1 -- 172.21.15.26:0/3989590904 shutdown_connections 2024-04-26T21:08:43.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.891+0000 7f544f200640 1 --2- 172.21.15.26:0/3989590904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5434004800 0x7f5434004c80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:43.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.891+0000 7f544f200640 1 -- 172.21.15.26:0/3989590904 >> 172.21.15.26:0/3989590904 conn(0x7f545009e410 msgr2=0x7f54500d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:43.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.891+0000 7f544f200640 1 -- 172.21.15.26:0/3989590904 shutdown_connections 2024-04-26T21:08:43.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.891+0000 7f544f200640 1 -- 172.21.15.26:0/3989590904 wait complete. 2024-04-26T21:08:43.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.893+0000 7f544f200640 1 Processor -- start 2024-04-26T21:08:43.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.893+0000 7f544f200640 1 -- start start 2024-04-26T21:08:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.894+0000 7f544f200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5434004800 0x7f54500b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.894+0000 7f544f200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54501576e0 0x7f545015e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.894+0000 7f544f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5450156f20 0x7f545015a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.894+0000 7f544f200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5450162b00 con 0x7f54501576e0 2024-04-26T21:08:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.894+0000 7f544f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5450162980 con 0x7f5450156f20 2024-04-26T21:08:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.894+0000 7f544f200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5450162c80 con 0x7f5434004800 2024-04-26T21:08:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.894+0000 7f5445400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54501576e0 0x7f545015e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.894+0000 7f5445400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54501576e0 0x7f545015e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34504/0 (socket says 172.21.15.26:34504) 2024-04-26T21:08:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.894+0000 7f5445400640 1 -- 172.21.15.26:0/903817006 learned_addr learned my addr 172.21.15.26:0/903817006 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.894+0000 7f544de00640 1 --2- 172.21.15.26:0/903817006 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5434004800 0x7f54500b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.894+0000 7f5445400640 1 -- 172.21.15.26:0/903817006 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5434004800 msgr2=0x7f54500b8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.894+0000 7f5445400640 1 --2- 172.21.15.26:0/903817006 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5434004800 0x7f54500b8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.894+0000 7f5445400640 1 -- 172.21.15.26:0/903817006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5450156f20 msgr2=0x7f545015a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.894+0000 7f544e800640 1 --2- 172.21.15.26:0/903817006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5450156f20 0x7f545015a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.895+0000 7f5445400640 1 --2- 172.21.15.26:0/903817006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5450156f20 0x7f545015a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.895+0000 7f5445400640 1 -- 172.21.15.26:0/903817006 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5450157b60 con 0x7f54501576e0 2024-04-26T21:08:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.895+0000 7f544de00640 1 --2- 172.21.15.26:0/903817006 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5434004800 0x7f54500b8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.895+0000 7f5445400640 1 --2- 172.21.15.26:0/903817006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54501576e0 0x7f545015e770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5438001870 tx=0x7f5438001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.895+0000 7f5446a00640 1 -- 172.21.15.26:0/903817006 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438037040 con 0x7f54501576e0 2024-04-26T21:08:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.895+0000 7f5446a00640 1 -- 172.21.15.26:0/903817006 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5438005480 con 0x7f54501576e0 2024-04-26T21:08:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.895+0000 7f544f200640 1 -- 172.21.15.26:0/903817006 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54500c9b10 con 0x7f54501576e0 2024-04-26T21:08:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.895+0000 7f5446a00640 1 -- 172.21.15.26:0/903817006 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438007a30 con 0x7f54501576e0 2024-04-26T21:08:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.895+0000 7f544f200640 1 -- 172.21.15.26:0/903817006 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54500ca090 con 0x7f54501576e0 2024-04-26T21:08:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.896+0000 7f544f200640 1 -- 172.21.15.26:0/903817006 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54500693d0 con 0x7f54501576e0 2024-04-26T21:08:43.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.897+0000 7f5446a00640 1 -- 172.21.15.26:0/903817006 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5438044020 con 0x7f54501576e0 2024-04-26T21:08:43.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.898+0000 7f5446a00640 1 --2- 172.21.15.26:0/903817006 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f542c06c8f0 0x7f542c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:43.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.898+0000 7f5446a00640 1 -- 172.21.15.26:0/903817006 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f54380ab8b0 con 0x7f54501576e0 2024-04-26T21:08:43.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.898+0000 7f544de00640 1 --2- 172.21.15.26:0/903817006 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f542c06c8f0 0x7f542c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:43.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.899+0000 7f544de00640 1 --2- 172.21.15.26:0/903817006 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f542c06c8f0 0x7f542c06edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5440005b70 tx=0x7f5440001290 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:43.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:43.903+0000 7f5446a00640 1 -- 172.21.15.26:0/903817006 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5450157b60 con 0x7f54501576e0 2024-04-26T21:08:44.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.101+0000 7f544f200640 1 -- 172.21.15.26:0/903817006 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f54500cbf20 con 0x7f54501576e0 2024-04-26T21:08:44.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.103+0000 7f5446a00640 1 -- 172.21.15.26:0/903817006 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16557 (secure 0 0 0) 0x7f5438079bb0 con 0x7f54501576e0 2024-04-26T21:08:44.104 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:44.104 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:08:39.177854+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:08:35.481169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:08:36.609498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:08:44.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.106+0000 7f544f200640 1 -- 172.21.15.26:0/903817006 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f542c06c8f0 msgr2=0x7f542c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.106+0000 7f544f200640 1 --2- 172.21.15.26:0/903817006 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f542c06c8f0 0x7f542c06edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5440005b70 tx=0x7f5440001290 comp rx=0 tx=0).stop 2024-04-26T21:08:44.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.106+0000 7f544f200640 1 -- 172.21.15.26:0/903817006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54501576e0 msgr2=0x7f545015e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.106+0000 7f544f200640 1 --2- 172.21.15.26:0/903817006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54501576e0 0x7f545015e770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5438001870 tx=0x7f5438001120 comp rx=0 tx=0).stop 2024-04-26T21:08:44.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.106+0000 7f544f200640 1 -- 172.21.15.26:0/903817006 shutdown_connections 2024-04-26T21:08:44.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.106+0000 7f544f200640 1 --2- 172.21.15.26:0/903817006 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f542c06c8f0 0x7f542c06edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.106+0000 7f544f200640 1 --2- 172.21.15.26:0/903817006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5450156f20 0x7f545015a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.106+0000 7f544f200640 1 --2- 172.21.15.26:0/903817006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54501576e0 0x7f545015e770 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.106+0000 7f544f200640 1 --2- 172.21.15.26:0/903817006 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5434004800 0x7f54500b8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.106+0000 7f544f200640 1 -- 172.21.15.26:0/903817006 >> 172.21.15.26:0/903817006 conn(0x7f545009e410 msgr2=0x7f54500d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:44.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.106+0000 7f544f200640 1 -- 172.21.15.26:0/903817006 shutdown_connections 2024-04-26T21:08:44.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.106+0000 7f544f200640 1 -- 172.21.15.26:0/903817006 wait complete. 2024-04-26T21:08:44.124 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-26T21:08:44.125 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:08:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.333+0000 7f5604200640 1 Processor -- start 2024-04-26T21:08:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.333+0000 7f5604200640 1 -- start start 2024-04-26T21:08:44.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.335+0000 7f5604200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55fc1458d0 0x7f55fc073360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.335+0000 7f5604200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f55fc0738a0 0x7f55fc0d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.335+0000 7f5604200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55fc15fd70 con 0x7f55fc0d1230 2024-04-26T21:08:44.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.335+0000 7f5604200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f55fc0d1680 con 0x7f55fc0738a0 2024-04-26T21:08:44.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.335+0000 7f5604200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f55fc160050 con 0x7f55fc1458d0 2024-04-26T21:08:44.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.335+0000 7f5602200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f55fc0d1230 0x7f55fc06f390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44718/0 (socket says 172.21.15.26:44718) 2024-04-26T21:08:44.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.335+0000 7f5602200640 1 -- 172.21.15.26:0/2604152546 learned_addr learned my addr 172.21.15.26:0/2604152546 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:44.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.336+0000 7f5601800640 1 --2- 172.21.15.26:0/2604152546 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55fc1458d0 0x7f55fc073360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.336+0000 7f5600e00640 1 --2- 172.21.15.26:0/2604152546 >> v2:172.21.15.86:3300/0 conn(0x7f55fc0738a0 0x7f55fc0d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.336+0000 7f55fbe00640 1 -- 172.21.15.26:0/2604152546 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3886171100 0 0) 0x7f55fc15fd70 con 0x7f55fc0d1230 2024-04-26T21:08:44.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.336+0000 7f55fbe00640 1 -- 172.21.15.26:0/2604152546 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55e4005730 con 0x7f55fc0d1230 2024-04-26T21:08:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.336+0000 7f55fbe00640 1 -- 172.21.15.26:0/2604152546 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 448099419 0 0) 0x7f55e4005730 con 0x7f55fc0d1230 2024-04-26T21:08:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.336+0000 7f55fbe00640 1 -- 172.21.15.26:0/2604152546 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55fc1458d0 msgr2=0x7f55fc073360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.336+0000 7f55fbe00640 1 --2- 172.21.15.26:0/2604152546 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55fc1458d0 0x7f55fc073360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.336+0000 7f55fbe00640 1 -- 172.21.15.26:0/2604152546 >> v2:172.21.15.86:3300/0 conn(0x7f55fc0738a0 msgr2=0x7f55fc0d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.336+0000 7f55fbe00640 1 --2- 172.21.15.26:0/2604152546 >> v2:172.21.15.86:3300/0 conn(0x7f55fc0738a0 0x7f55fc0d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.336+0000 7f55fbe00640 1 -- 172.21.15.26:0/2604152546 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55fc160330 con 0x7f55fc0d1230 2024-04-26T21:08:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.336+0000 7f55fbe00640 1 -- 172.21.15.26:0/2604152546 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f55f40019f0 con 0x7f55fc0d1230 2024-04-26T21:08:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.336+0000 7f5600e00640 1 --2- 172.21.15.26:0/2604152546 >> v2:172.21.15.86:3300/0 conn(0x7f55fc0738a0 0x7f55fc0d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:08:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.336+0000 7f55fbe00640 1 -- 172.21.15.26:0/2604152546 >> v1:172.21.15.26:6789/0 conn(0x7f55fc0d1230 legacy=0x7f55fc06f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.336+0000 7f55fbe00640 1 --2- 172.21.15.26:0/2604152546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e4005bc0 0x7f55e4005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.336+0000 7f55fbe00640 1 --2- 172.21.15.26:0/2604152546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e4006a10 0x7f55e4006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.336+0000 7f55fbe00640 1 --2- 172.21.15.26:0/2604152546 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55fc1458d0 0x7f55e40073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.336+0000 7f55fbe00640 1 -- 172.21.15.26:0/2604152546 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc0d1680 con 0x7f55e4005bc0 2024-04-26T21:08:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.336+0000 7f55fbe00640 1 -- 172.21.15.26:0/2604152546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc160050 con 0x7f55e4006a10 2024-04-26T21:08:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.336+0000 7f55fbe00640 1 -- 172.21.15.26:0/2604152546 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f55e400dc20 con 0x7f55fc1458d0 2024-04-26T21:08:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.337+0000 7f5601800640 1 --2- 172.21.15.26:0/2604152546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e4005bc0 0x7f55e4005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.337+0000 7f5600e00640 1 --2- 172.21.15.26:0/2604152546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e4006a10 0x7f55e4006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.337+0000 7f5602200640 1 --2- 172.21.15.26:0/2604152546 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55fc1458d0 0x7f55e40073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.337+0000 7f5601800640 1 -- 172.21.15.26:0/2604152546 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55fc1458d0 msgr2=0x7f55e40073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.337+0000 7f5601800640 1 --2- 172.21.15.26:0/2604152546 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55fc1458d0 0x7f55e40073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.337+0000 7f5601800640 1 -- 172.21.15.26:0/2604152546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e4006a10 msgr2=0x7f55e4006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.337+0000 7f5601800640 1 --2- 172.21.15.26:0/2604152546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e4006a10 0x7f55e4006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.337+0000 7f5601800640 1 -- 172.21.15.26:0/2604152546 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e4005730 con 0x7f55e4005bc0 2024-04-26T21:08:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.337+0000 7f5600e00640 1 --2- 172.21.15.26:0/2604152546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e4006a10 0x7f55e4006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.337+0000 7f5602200640 1 --2- 172.21.15.26:0/2604152546 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55fc1458d0 0x7f55e40073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:44.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.338+0000 7f5601800640 1 --2- 172.21.15.26:0/2604152546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e4005bc0 0x7f55e4005fe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009ac0 tx=0x7f55e4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637c173741c18930 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.338+0000 7f55fbe00640 1 -- 172.21.15.26:0/2604152546 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0036070 con 0x7f55e4005bc0 2024-04-26T21:08:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.339+0000 7f55fbe00640 1 -- 172.21.15.26:0/2604152546 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f0031400 con 0x7f55e4005bc0 2024-04-26T21:08:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.339+0000 7f55fbe00640 1 -- 172.21.15.26:0/2604152546 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0009690 con 0x7f55e4005bc0 2024-04-26T21:08:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.339+0000 7f5604200640 1 -- 172.21.15.26:0/2604152546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e4005bc0 msgr2=0x7f55e4005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.339+0000 7f5604200640 1 --2- 172.21.15.26:0/2604152546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e4005bc0 0x7f55e4005fe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009ac0 tx=0x7f55e4007a40 comp rx=0 tx=0).stop 2024-04-26T21:08:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.339+0000 7f5602200640 1 -- 172.21.15.26:0/2604152546 reap_dead start 2024-04-26T21:08:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.339+0000 7f5604200640 1 -- 172.21.15.26:0/2604152546 shutdown_connections 2024-04-26T21:08:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.339+0000 7f5604200640 1 -- 172.21.15.26:0/2604152546 >> 172.21.15.26:0/2604152546 conn(0x7f55fc0cd840 msgr2=0x7f55fc0cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.339+0000 7f5604200640 1 -- 172.21.15.26:0/2604152546 shutdown_connections 2024-04-26T21:08:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.339+0000 7f5604200640 1 -- 172.21.15.26:0/2604152546 wait complete. 2024-04-26T21:08:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.341+0000 7f5604200640 1 Processor -- start 2024-04-26T21:08:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.341+0000 7f5604200640 1 -- start start 2024-04-26T21:08:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.342+0000 7f5604200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e4005bc0 0x7f55fc144b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.342+0000 7f5604200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55fc13b140 0x7f55fc0b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.342+0000 7f5604200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55fc0b1dc0 0x7f55fc15dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.342+0000 7f5604200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc145cb0 con 0x7f55fc0b1dc0 2024-04-26T21:08:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.342+0000 7f5604200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc1608f0 con 0x7f55e4005bc0 2024-04-26T21:08:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.342+0000 7f5604200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f55fc072f90 con 0x7f55fc13b140 2024-04-26T21:08:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.342+0000 7f5602200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55fc0b1dc0 0x7f55fc15dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.343+0000 7f5602200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55fc0b1dc0 0x7f55fc15dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34524/0 (socket says 172.21.15.26:34524) 2024-04-26T21:08:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.343+0000 7f5602200640 1 -- 172.21.15.26:0/1069623433 learned_addr learned my addr 172.21.15.26:0/1069623433 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.343+0000 7f5600e00640 1 --2- 172.21.15.26:0/1069623433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55fc13b140 0x7f55fc0b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.343+0000 7f5601800640 1 --2- 172.21.15.26:0/1069623433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e4005bc0 0x7f55fc144b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.343+0000 7f5602200640 1 -- 172.21.15.26:0/1069623433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55fc13b140 msgr2=0x7f55fc0b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.343+0000 7f5602200640 1 --2- 172.21.15.26:0/1069623433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55fc13b140 0x7f55fc0b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.343+0000 7f5602200640 1 -- 172.21.15.26:0/1069623433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e4005bc0 msgr2=0x7f55fc144b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.343+0000 7f5602200640 1 --2- 172.21.15.26:0/1069623433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e4005bc0 0x7f55fc144b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.343+0000 7f5602200640 1 -- 172.21.15.26:0/1069623433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55fc13b610 con 0x7f55fc0b1dc0 2024-04-26T21:08:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.343+0000 7f5601800640 1 --2- 172.21.15.26:0/1069623433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e4005bc0 0x7f55fc144b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.343+0000 7f5602200640 1 --2- 172.21.15.26:0/1069623433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55fc0b1dc0 0x7f55fc15dbe0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f55fc070410 tx=0x7f55f400df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.343+0000 7f55f9600640 1 -- 172.21.15.26:0/1069623433 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f400b860 con 0x7f55fc0b1dc0 2024-04-26T21:08:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.344+0000 7f55f9600640 1 -- 172.21.15.26:0/1069623433 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f400ba00 con 0x7f55fc0b1dc0 2024-04-26T21:08:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.344+0000 7f55f9600640 1 -- 172.21.15.26:0/1069623433 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f400bdf0 con 0x7f55fc0b1dc0 2024-04-26T21:08:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.344+0000 7f5604200640 1 -- 172.21.15.26:0/1069623433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55fc13b8f0 con 0x7f55fc0b1dc0 2024-04-26T21:08:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.344+0000 7f5604200640 1 -- 172.21.15.26:0/1069623433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55fc13beb0 con 0x7f55fc0b1dc0 2024-04-26T21:08:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.345+0000 7f5604200640 1 -- 172.21.15.26:0/1069623433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55fc06a700 con 0x7f55fc0b1dc0 2024-04-26T21:08:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.346+0000 7f55f9600640 1 -- 172.21.15.26:0/1069623433 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f55f401b020 con 0x7f55fc0b1dc0 2024-04-26T21:08:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.347+0000 7f55f9600640 1 --2- 172.21.15.26:0/1069623433 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f55dc06c940 0x7f55dc06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.348+0000 7f55f9600640 1 -- 172.21.15.26:0/1069623433 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f55f408bb40 con 0x7f55fc0b1dc0 2024-04-26T21:08:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.348+0000 7f5601800640 1 --2- 172.21.15.26:0/1069623433 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f55dc06c940 0x7f55dc06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.349+0000 7f5601800640 1 --2- 172.21.15.26:0/1069623433 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f55dc06c940 0x7f55dc06ee00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f55f0033000 tx=0x7f55f0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.352+0000 7f55f9600640 1 -- 172.21.15.26:0/1069623433 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f55f4059fe0 con 0x7f55fc0b1dc0 2024-04-26T21:08:44.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.560+0000 7f5604200640 1 -- 172.21.15.26:0/1069623433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55fc0b6c90 con 0x7f55fc0b1dc0 2024-04-26T21:08:44.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.563+0000 7f55f9600640 1 -- 172.21.15.26:0/1069623433 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16557 (secure 0 0 0) 0x7f55f405ee20 con 0x7f55fc0b1dc0 2024-04-26T21:08:44.564 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:44.564 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:08:39.177854+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:08:35.481169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:08:36.609498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:08:44.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.565+0000 7f5604200640 1 -- 172.21.15.26:0/1069623433 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f55dc06c940 msgr2=0x7f55dc06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.565+0000 7f5604200640 1 --2- 172.21.15.26:0/1069623433 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f55dc06c940 0x7f55dc06ee00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f55f0033000 tx=0x7f55f0008030 comp rx=0 tx=0).stop 2024-04-26T21:08:44.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.565+0000 7f5604200640 1 -- 172.21.15.26:0/1069623433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55fc0b1dc0 msgr2=0x7f55fc15dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.565+0000 7f5604200640 1 --2- 172.21.15.26:0/1069623433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55fc0b1dc0 0x7f55fc15dbe0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f55fc070410 tx=0x7f55f400df60 comp rx=0 tx=0).stop 2024-04-26T21:08:44.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.566+0000 7f5604200640 1 -- 172.21.15.26:0/1069623433 shutdown_connections 2024-04-26T21:08:44.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.566+0000 7f5604200640 1 --2- 172.21.15.26:0/1069623433 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f55dc06c940 0x7f55dc06ee00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.566+0000 7f5604200640 1 --2- 172.21.15.26:0/1069623433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55fc0b1dc0 0x7f55fc15dbe0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.566+0000 7f5604200640 1 --2- 172.21.15.26:0/1069623433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55fc13b140 0x7f55fc0b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.566+0000 7f5604200640 1 --2- 172.21.15.26:0/1069623433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e4005bc0 0x7f55fc144b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.566+0000 7f5604200640 1 -- 172.21.15.26:0/1069623433 >> 172.21.15.26:0/1069623433 conn(0x7f55fc0cd840 msgr2=0x7f55fc072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:44.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.566+0000 7f5604200640 1 -- 172.21.15.26:0/1069623433 shutdown_connections 2024-04-26T21:08:44.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.566+0000 7f5604200640 1 -- 172.21.15.26:0/1069623433 wait complete. 2024-04-26T21:08:44.587 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T21:08:44.587 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-26T21:08:44.588 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T21:08:44.588 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-26T21:08:44.588 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-26T21:08:44.589 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-26T21:08:44.589 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-26T21:08:44.589 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-26T21:08:44.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.891+0000 7fed95000640 1 Processor -- start 2024-04-26T21:08:44.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.891+0000 7fed95000640 1 -- start start 2024-04-26T21:08:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.893+0000 7fed95000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fed90163730 0x7fed900d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.893+0000 7fed95000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed900b3df0 0x7fed900d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.893+0000 7fed95000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed9006e280 con 0x7fed90167180 2024-04-26T21:08:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.893+0000 7fed95000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fed90167680 con 0x7fed90163730 2024-04-26T21:08:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.893+0000 7fed95000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fed90167960 con 0x7fed900b3df0 2024-04-26T21:08:44.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.894+0000 7fed8de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fed90163730 0x7fed900d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.895+0000 7fed8de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fed90163730 0x7fed900d0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.26:41668/0 (socket says 172.21.15.26:41668) 2024-04-26T21:08:44.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.895+0000 7fed8de00640 1 -- 172.21.15.26:0/1259635046 learned_addr learned my addr 172.21.15.26:0/1259635046 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:44.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.894+0000 7fed8e800640 1 --1- 172.21.15.26:0/1259635046 >> v1:172.21.15.26:6789/0 conn(0x7fed90167180 0x7fed900ce950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44732/0 (socket says 172.21.15.26:44732) 2024-04-26T21:08:44.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.895+0000 7fed8d400640 1 --2- 172.21.15.26:0/1259635046 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed900b3df0 0x7fed900d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.895+0000 7fe72fe00640 1 Processor -- start 2024-04-26T21:08:44.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.895+0000 7f28d6c00640 1 Processor -- start 2024-04-26T21:08:44.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.896+0000 7f28d6c00640 1 -- start start 2024-04-26T21:08:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.896+0000 7fe72fe00640 1 -- start start 2024-04-26T21:08:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.897+0000 7fe72fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe7280d39f0 0x7fe7280d5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.897+0000 7fe72fe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7280d3dd0 0x7fe7280d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.897+0000 7fe72fe00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7280d0ef0 con 0x7fe72815f000 2024-04-26T21:08:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.897+0000 7fe72fe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe7280b4900 con 0x7fe7280d39f0 2024-04-26T21:08:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.897+0000 7fe72fe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe7280b4be0 con 0x7fe7280d3dd0 2024-04-26T21:08:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.898+0000 7f28d6c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f28d00b4470 0x7f28d00d3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.898+0000 7f28d6c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28d00d4000 0x7f28d00d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.896+0000 7fed8ca00640 1 -- 172.21.15.26:0/1259635046 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 958921699 0 0) 0x7fed9006e280 con 0x7fed90167180 2024-04-26T21:08:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.896+0000 7fed8ca00640 1 -- 172.21.15.26:0/1259635046 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed74003520 con 0x7fed90167180 2024-04-26T21:08:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.897+0000 7fed8d400640 1 -- 172.21.15.26:0/1259635046 >> v2:172.21.15.86:3300/0 conn(0x7fed90163730 msgr2=0x7fed900d0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.897+0000 7fed8d400640 1 --2- 172.21.15.26:0/1259635046 >> v2:172.21.15.86:3300/0 conn(0x7fed90163730 0x7fed900d0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.897+0000 7fed8d400640 1 -- 172.21.15.26:0/1259635046 >> v1:172.21.15.26:6789/0 conn(0x7fed90167180 legacy=0x7fed900ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.897+0000 7fed8d400640 1 -- 172.21.15.26:0/1259635046 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed90167c40 con 0x7fed900b3df0 2024-04-26T21:08:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.899+0000 7fe72d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe7280d39f0 0x7fe7280d5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.899+0000 7fe72ca00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7280d3dd0 0x7fe7280d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.899+0000 7fe72de00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe72815f000 0x7fe72815f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44734/0 (socket says 172.21.15.26:44734) 2024-04-26T21:08:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.899+0000 7fe72de00640 1 -- 172.21.15.26:0/2834167951 learned_addr learned my addr 172.21.15.26:0/2834167951 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.899+0000 7f28d6c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28d00b4850 con 0x7f28d00cfd30 2024-04-26T21:08:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.899+0000 7f28d6c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f28d0162fd0 con 0x7f28d00d4000 2024-04-26T21:08:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.899+0000 7f28d6c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f28d00d0f00 con 0x7f28d00b4470 2024-04-26T21:08:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.900+0000 7f28cfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f28d00b4470 0x7f28d00d3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.900+0000 7f28cf400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28d00d4000 0x7f28d00d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.900+0000 7fed8d400640 1 --2- 172.21.15.26:0/1259635046 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed900b3df0 0x7fed900d0cd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0097a0 tx=0x7fed7c02a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=775f623150d37980 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.900+0000 7f28cfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f28d00b4470 0x7f28d00d3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.26:41682/0 (socket says 172.21.15.26:41682) 2024-04-26T21:08:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.900+0000 7f28cfe00640 1 -- 172.21.15.26:0/4180225701 learned_addr learned my addr 172.21.15.26:0/4180225701 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:44.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.900+0000 7fe72d400640 1 -- 172.21.15.26:0/2834167951 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7280d3dd0 msgr2=0x7fe7280d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:44.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.900+0000 7fe72d400640 1 --2- 172.21.15.26:0/2834167951 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7280d3dd0 0x7fe7280d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.900+0000 7fe72d400640 1 -- 172.21.15.26:0/2834167951 >> v1:172.21.15.26:6789/0 conn(0x7fe72815f000 legacy=0x7fe72815f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.900+0000 7fe727e00640 1 -- 172.21.15.26:0/2834167951 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 929078558 0 0) 0x7fe7280d0ef0 con 0x7fe72815f000 2024-04-26T21:08:44.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.900+0000 7fed8ca00640 1 -- 172.21.15.26:0/1259635046 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c037070 con 0x7fed900b3df0 2024-04-26T21:08:44.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.901+0000 7fed8ca00640 1 -- 172.21.15.26:0/1259635046 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed7c032440 con 0x7fed900b3df0 2024-04-26T21:08:44.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.901+0000 7fed8ca00640 1 -- 172.21.15.26:0/1259635046 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c032830 con 0x7fed900b3df0 2024-04-26T21:08:44.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.901+0000 7f28cea00640 1 -- 172.21.15.26:0/4180225701 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1046283251 0 0) 0x7f28d00b4850 con 0x7f28d00cfd30 2024-04-26T21:08:44.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.900+0000 7fe72d400640 1 -- 172.21.15.26:0/2834167951 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7280b4ec0 con 0x7fe7280d39f0 2024-04-26T21:08:44.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.901+0000 7f28cea00640 1 -- 172.21.15.26:0/4180225701 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28b4003520 con 0x7f28d00cfd30 2024-04-26T21:08:44.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.901+0000 7f28cf400640 1 -- 172.21.15.26:0/4180225701 >> v2:172.21.15.86:3300/0 conn(0x7f28d00b4470 msgr2=0x7f28d00d3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:44.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.901+0000 7f28cf400640 1 --2- 172.21.15.26:0/4180225701 >> v2:172.21.15.86:3300/0 conn(0x7f28d00b4470 0x7f28d00d3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.901+0000 7f28cf400640 1 -- 172.21.15.26:0/4180225701 >> v1:172.21.15.26:6789/0 conn(0x7f28d00cfd30 legacy=0x7f28d01631e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.901+0000 7f28cf400640 1 -- 172.21.15.26:0/4180225701 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d00d02f0 con 0x7f28d00d4000 2024-04-26T21:08:44.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.902+0000 7fed95000640 1 -- 172.21.15.26:0/1259635046 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed900b3df0 msgr2=0x7fed900d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.902+0000 7fed95000640 1 --2- 172.21.15.26:0/1259635046 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed900b3df0 0x7fed900d0cd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0097a0 tx=0x7fed7c02a130 comp rx=0 tx=0).stop 2024-04-26T21:08:44.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.902+0000 7f143c600640 1 Processor -- start 2024-04-26T21:08:44.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.902+0000 7fed95000640 1 -- 172.21.15.26:0/1259635046 shutdown_connections 2024-04-26T21:08:44.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.902+0000 7fed95000640 1 --2- 172.21.15.26:0/1259635046 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed900b3df0 0x7fed900d0cd0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.902+0000 7fed95000640 1 --2- 172.21.15.26:0/1259635046 >> v2:172.21.15.86:3300/0 conn(0x7fed90163730 0x7fed900d0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.902+0000 7fed95000640 1 -- 172.21.15.26:0/1259635046 >> 172.21.15.26:0/1259635046 conn(0x7fed900cd840 msgr2=0x7fed900cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:44.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.902+0000 7f143c600640 1 -- start start 2024-04-26T21:08:44.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.902+0000 7fed95000640 1 -- 172.21.15.26:0/1259635046 shutdown_connections 2024-04-26T21:08:44.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.902+0000 7fed95000640 1 -- 172.21.15.26:0/1259635046 wait complete. 2024-04-26T21:08:44.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.903+0000 7fe72d400640 1 --2- 172.21.15.26:0/2834167951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7280d39f0 0x7fe7280d5cc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0097a0 tx=0x7fe71c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ec2e8c80177ffcb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.903+0000 7fe727e00640 1 -- 172.21.15.26:0/2834167951 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c037070 con 0x7fe7280d39f0 2024-04-26T21:08:44.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.903+0000 7fedf5800640 1 Processor -- start 2024-04-26T21:08:44.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.903+0000 7fedf5800640 1 -- start start 2024-04-26T21:08:44.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.903+0000 7f28cf400640 1 --2- 172.21.15.26:0/4180225701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28d00d4000 0x7f28d00d18c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f28d008eaa0 tx=0x7f28c002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2ee7e5443c4a23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.903+0000 7fe727e00640 1 -- 172.21.15.26:0/2834167951 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe71c032440 con 0x7fe7280d39f0 2024-04-26T21:08:44.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.903+0000 7fe727e00640 1 -- 172.21.15.26:0/2834167951 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c032810 con 0x7fe7280d39f0 2024-04-26T21:08:44.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.904+0000 7f143c600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f143415f130 0x7f14340b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.905+0000 7fed95000640 1 Processor -- start 2024-04-26T21:08:44.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.904+0000 7f143c600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f143415f510 0x7f143406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.904+0000 7f143c600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f143406e490 con 0x7f14340cf710 2024-04-26T21:08:44.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.905+0000 7f143c600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f14340cf350 con 0x7f143415f130 2024-04-26T21:08:44.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.905+0000 7f143c600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f14340cd580 con 0x7f143415f510 2024-04-26T21:08:44.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.905+0000 7f28cea00640 1 -- 172.21.15.26:0/4180225701 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0037070 con 0x7f28d00d4000 2024-04-26T21:08:44.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.905+0000 7f28cea00640 1 -- 172.21.15.26:0/4180225701 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28c0032440 con 0x7f28d00d4000 2024-04-26T21:08:44.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.905+0000 7f28cea00640 1 -- 172.21.15.26:0/4180225701 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0032810 con 0x7f28d00d4000 2024-04-26T21:08:44.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.905+0000 7fe72fe00640 1 -- 172.21.15.26:0/2834167951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7280d39f0 msgr2=0x7fe7280d5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.905+0000 7fe72fe00640 1 --2- 172.21.15.26:0/2834167951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7280d39f0 0x7fe7280d5cc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0097a0 tx=0x7fe71c02a180 comp rx=0 tx=0).stop 2024-04-26T21:08:44.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.905+0000 7fed95000640 1 -- start start 2024-04-26T21:08:44.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.905+0000 7f1439200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f143415f510 0x7f143406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.905+0000 7f1439200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f143415f510 0x7f143406dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.26:45918/0 (socket says 172.21.15.26:45918) 2024-04-26T21:08:44.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.905+0000 7f1439200640 1 -- 172.21.15.26:0/3280875553 learned_addr learned my addr 172.21.15.26:0/3280875553 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:44.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.906+0000 7fe72fe00640 1 -- 172.21.15.26:0/2834167951 shutdown_connections 2024-04-26T21:08:44.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.906+0000 7fe72fe00640 1 --2- 172.21.15.26:0/2834167951 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7280d3dd0 0x7fe7280d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.906+0000 7fe72fe00640 1 --2- 172.21.15.26:0/2834167951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7280d39f0 0x7fe7280d5cc0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.906+0000 7fe72fe00640 1 -- 172.21.15.26:0/2834167951 >> 172.21.15.26:0/2834167951 conn(0x7fe7280d51a0 msgr2=0x7fe7280d55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:44.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.906+0000 7f28d6c00640 1 -- 172.21.15.26:0/4180225701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28d00d4000 msgr2=0x7f28d00d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.906+0000 7f28d6c00640 1 --2- 172.21.15.26:0/4180225701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28d00d4000 0x7f28d00d18c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f28d008eaa0 tx=0x7f28c002a230 comp rx=0 tx=0).stop 2024-04-26T21:08:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.906+0000 7f1439c00640 1 --2- 172.21.15.26:0/3280875553 >> v2:172.21.15.26:3300/0 conn(0x7f143415f130 0x7f14340b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.906+0000 7fe72fe00640 1 -- 172.21.15.26:0/2834167951 shutdown_connections 2024-04-26T21:08:44.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.907+0000 7fed95000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed900b3df0 0x7fed90165380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.907+0000 7fed95000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed90163730 0x7fed9014a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.908+0000 7fed8d400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed90163730 0x7fed9014a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.907+0000 7f1433e00640 1 -- 172.21.15.26:0/3280875553 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4084833326 0 0) 0x7f143406e490 con 0x7f14340cf710 2024-04-26T21:08:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.907+0000 7f1433e00640 1 -- 172.21.15.26:0/3280875553 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f141c003520 con 0x7f14340cf710 2024-04-26T21:08:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.908+0000 7f1439200640 1 -- 172.21.15.26:0/3280875553 >> v2:172.21.15.26:3300/0 conn(0x7f143415f130 msgr2=0x7f14340b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.908+0000 7f1439200640 1 --2- 172.21.15.26:0/3280875553 >> v2:172.21.15.26:3300/0 conn(0x7f143415f130 0x7f14340b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.908+0000 7f1439200640 1 -- 172.21.15.26:0/3280875553 >> v1:172.21.15.26:6789/0 conn(0x7f14340cf710 legacy=0x7f14340b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.908+0000 7f1439200640 1 -- 172.21.15.26:0/3280875553 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14340cd860 con 0x7f143415f510 2024-04-26T21:08:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.908+0000 7fed95000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed90167180 0x7fed9015c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.908+0000 7fed95000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fed900d0ad0 con 0x7fed90163730 2024-04-26T21:08:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.908+0000 7fed95000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed900d0950 con 0x7fed90167180 2024-04-26T21:08:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.908+0000 7fed95000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fed900d1210 con 0x7fed900b3df0 2024-04-26T21:08:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.908+0000 7fed8de00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed900b3df0 0x7fed90165380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.908+0000 7fed8de00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed900b3df0 0x7fed90165380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.26:45922/0 (socket says 172.21.15.26:45922) 2024-04-26T21:08:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.908+0000 7fed8de00640 1 -- 172.21.15.26:0/314025201 learned_addr learned my addr 172.21.15.26:0/314025201 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.908+0000 7f1439c00640 1 --2- 172.21.15.26:0/3280875553 >> v2:172.21.15.26:3300/0 conn(0x7f143415f130 0x7f14340b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:08:44.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.906+0000 7fedf5800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fedf0070120 0x7fedf015e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.906+0000 7fedf5800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fedf0073d30 0x7fedf00ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.907+0000 7fedf5800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedf006fca0 con 0x7fedf00b3d10 2024-04-26T21:08:44.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.907+0000 7fedf5800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fedf00721b0 con 0x7fedf0073d30 2024-04-26T21:08:44.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.907+0000 7fedf5800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fedf0072490 con 0x7fedf0070120 2024-04-26T21:08:44.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.907+0000 7fedf4e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fedf00b3d10 0x7fedf00b2f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44770/0 (socket says 172.21.15.26:44770) 2024-04-26T21:08:44.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.907+0000 7fedf4e00640 1 -- 172.21.15.26:0/2867511279 learned_addr learned my addr 172.21.15.26:0/2867511279 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:44.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.906+0000 7fe72fe00640 1 -- 172.21.15.26:0/2834167951 wait complete. 2024-04-26T21:08:44.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.907+0000 7fedefe00640 1 --2- 172.21.15.26:0/2867511279 >> v2:172.21.15.86:3300/0 conn(0x7fedf0070120 0x7fedf015e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.908+0000 7fedef400640 1 --2- 172.21.15.26:0/2867511279 >> v2:172.21.15.26:3300/0 conn(0x7fedf0073d30 0x7fedf00ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.908+0000 7fedefe00640 1 -- 172.21.15.26:0/2867511279 >> v2:172.21.15.26:3300/0 conn(0x7fedf0073d30 msgr2=0x7fedf00ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:44.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.909+0000 7fed8d400640 1 -- 172.21.15.26:0/314025201 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed900b3df0 msgr2=0x7fed90165380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.909+0000 7fed8d400640 1 --2- 172.21.15.26:0/314025201 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed900b3df0 0x7fed90165380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.909+0000 7fed8d400640 1 -- 172.21.15.26:0/314025201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed90167180 msgr2=0x7fed9015c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.909+0000 7fed8d400640 1 --2- 172.21.15.26:0/314025201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed90167180 0x7fed9015c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.908+0000 7fedefe00640 1 --2- 172.21.15.26:0/2867511279 >> v2:172.21.15.26:3300/0 conn(0x7fedf0073d30 0x7fedf00ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.908+0000 7fedefe00640 1 -- 172.21.15.26:0/2867511279 >> v1:172.21.15.26:6789/0 conn(0x7fedf00b3d10 legacy=0x7fedf00b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.909+0000 7fedeea00640 1 -- 172.21.15.26:0/2867511279 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3407897599 0 0) 0x7fedf006fca0 con 0x7fedf00b3d10 2024-04-26T21:08:44.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.909+0000 7fedefe00640 1 -- 172.21.15.26:0/2867511279 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf0072770 con 0x7fedf0070120 2024-04-26T21:08:44.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.909+0000 7fed8d400640 1 -- 172.21.15.26:0/314025201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed901645c0 con 0x7fed90163730 2024-04-26T21:08:44.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.910+0000 7fed8d400640 1 --2- 172.21.15.26:0/314025201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed90163730 0x7fed9014a030 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fed900d1a10 tx=0x7fed7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.910+0000 7f28d6c00640 1 -- 172.21.15.26:0/4180225701 shutdown_connections 2024-04-26T21:08:44.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.910+0000 7f28d6c00640 1 --2- 172.21.15.26:0/4180225701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28d00d4000 0x7f28d00d18c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.910+0000 7f28d6c00640 1 --2- 172.21.15.26:0/4180225701 >> v2:172.21.15.86:3300/0 conn(0x7f28d00b4470 0x7f28d00d3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.910+0000 7f28d6c00640 1 -- 172.21.15.26:0/4180225701 >> 172.21.15.26:0/4180225701 conn(0x7f28d009e490 msgr2=0x7f28d00d51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:44.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.910+0000 7fed86000640 1 -- 172.21.15.26:0/314025201 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c041030 con 0x7fed90163730 2024-04-26T21:08:44.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.910+0000 7fed95000640 1 -- 172.21.15.26:0/314025201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed901648a0 con 0x7fed90163730 2024-04-26T21:08:44.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.911+0000 7fed95000640 1 -- 172.21.15.26:0/314025201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed90164e60 con 0x7fed90163730 2024-04-26T21:08:44.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.911+0000 7f28d6c00640 1 -- 172.21.15.26:0/4180225701 shutdown_connections 2024-04-26T21:08:44.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.911+0000 7fed86000640 1 -- 172.21.15.26:0/314025201 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed7c03d4e0 con 0x7fed90163730 2024-04-26T21:08:44.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.911+0000 7fed86000640 1 -- 172.21.15.26:0/314025201 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c041030 con 0x7fed90163730 2024-04-26T21:08:44.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.910+0000 7fe72fe00640 1 Processor -- start 2024-04-26T21:08:44.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.911+0000 7f28d6c00640 1 -- 172.21.15.26:0/4180225701 wait complete. 2024-04-26T21:08:44.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.911+0000 7fedefe00640 1 --2- 172.21.15.26:0/2867511279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf0070120 0x7fedf015e870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fedf006ef30 tx=0x7fede003b1b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5055844a7b6bb609 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.911+0000 7fedeea00640 1 -- 172.21.15.26:0/2867511279 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0048070 con 0x7fedf0070120 2024-04-26T21:08:44.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.911+0000 7fedeea00640 1 -- 172.21.15.26:0/2867511279 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fede0043440 con 0x7fedf0070120 2024-04-26T21:08:44.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.911+0000 7fedeea00640 1 -- 172.21.15.26:0/2867511279 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0043810 con 0x7fedf0070120 2024-04-26T21:08:44.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.912+0000 7f8e57200640 1 Processor -- start 2024-04-26T21:08:44.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.912+0000 7f1439200640 1 --2- 172.21.15.26:0/3280875553 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f143415f510 0x7f143406dd40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1420009870 tx=0x7f142002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f959e08011e036e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.913+0000 7f1433e00640 1 -- 172.21.15.26:0/3280875553 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420037070 con 0x7f143415f510 2024-04-26T21:08:44.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.912+0000 7f8e57200640 1 -- start start 2024-04-26T21:08:44.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.912+0000 7fedf5800640 1 -- 172.21.15.26:0/2867511279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf0070120 msgr2=0x7fedf015e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.912+0000 7fedf5800640 1 --2- 172.21.15.26:0/2867511279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf0070120 0x7fedf015e870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fedf006ef30 tx=0x7fede003b1b0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.913+0000 7fedf5800640 1 -- 172.21.15.26:0/2867511279 shutdown_connections 2024-04-26T21:08:44.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.913+0000 7fedf5800640 1 --2- 172.21.15.26:0/2867511279 >> v2:172.21.15.26:3300/0 conn(0x7fedf0073d30 0x7fedf00ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.913+0000 7fedf5800640 1 --2- 172.21.15.26:0/2867511279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf0070120 0x7fedf015e870 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.913+0000 7fedf5800640 1 -- 172.21.15.26:0/2867511279 >> 172.21.15.26:0/2867511279 conn(0x7fedf006c660 msgr2=0x7fedf006ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:44.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.914+0000 7fedf5800640 1 -- 172.21.15.26:0/2867511279 shutdown_connections 2024-04-26T21:08:44.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.914+0000 7fedf5800640 1 -- 172.21.15.26:0/2867511279 wait complete. 2024-04-26T21:08:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.914+0000 7fed95000640 1 -- 172.21.15.26:0/314025201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fed68001030 con 0x7fed90163730 2024-04-26T21:08:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.913+0000 7f1433e00640 1 -- 172.21.15.26:0/3280875553 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1420032440 con 0x7f143415f510 2024-04-26T21:08:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.913+0000 7f1433e00640 1 -- 172.21.15.26:0/3280875553 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420032810 con 0x7f143415f510 2024-04-26T21:08:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.914+0000 7f143c600640 1 -- 172.21.15.26:0/3280875553 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f143415f510 msgr2=0x7f143406dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.914+0000 7f143c600640 1 --2- 172.21.15.26:0/3280875553 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f143415f510 0x7f143406dd40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1420009870 tx=0x7f142002a250 comp rx=0 tx=0).stop 2024-04-26T21:08:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.912+0000 7fe72fe00640 1 -- start start 2024-04-26T21:08:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.913+0000 7fe72fe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7280d39f0 0x7fe728160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.913+0000 7fe72fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7280d3dd0 0x7fe7281f9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.914+0000 7fe72fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe72815f000 0x7fe7281f3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.914+0000 7fe72fe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe7280b5770 con 0x7fe7280d3dd0 2024-04-26T21:08:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.914+0000 7fe72fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7280b55f0 con 0x7fe72815f000 2024-04-26T21:08:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.914+0000 7fe72fe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe7280b58f0 con 0x7fe7280d39f0 2024-04-26T21:08:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.914+0000 7fe72d400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7280d39f0 0x7fe728160ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.914+0000 7fe72d400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7280d39f0 0x7fe728160ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.26:45930/0 (socket says 172.21.15.26:45930) 2024-04-26T21:08:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.914+0000 7fe72d400640 1 -- 172.21.15.26:0/1605825689 learned_addr learned my addr 172.21.15.26:0/1605825689 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.914+0000 7fe72ca00640 1 --2- 172.21.15.26:0/1605825689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7280d3dd0 0x7fe7281f9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.914+0000 7f8e57200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8e5015f710 0x7f8e500d5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.914+0000 7f8e57200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e500d0ef0 con 0x7f8e500d39f0 2024-04-26T21:08:44.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.914+0000 7f8e57200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e500b4900 con 0x7f8e500d3dd0 2024-04-26T21:08:44.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.914+0000 7f8e57200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f8e500b4be0 con 0x7f8e5015f710 2024-04-26T21:08:44.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.915+0000 7f143c600640 1 -- 172.21.15.26:0/3280875553 shutdown_connections 2024-04-26T21:08:44.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.915+0000 7f143c600640 1 --2- 172.21.15.26:0/3280875553 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f143415f510 0x7f143406dd40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.915+0000 7f8e4f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8e500d3dd0 0x7f8e500b53e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.26:33978/0 (socket says 172.21.15.26:33978) 2024-04-26T21:08:44.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.915+0000 7f8e4f400640 1 -- 172.21.15.26:0/121753664 learned_addr learned my addr 172.21.15.26:0/121753664 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:44.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.915+0000 7f8e55200640 1 --2- 172.21.15.26:0/121753664 >> v2:172.21.15.26:3300/0 conn(0x7f8e5015f710 0x7f8e500d5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.915+0000 7f143c600640 1 --2- 172.21.15.26:0/3280875553 >> v2:172.21.15.26:3300/0 conn(0x7f143415f130 0x7f14340b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.915+0000 7f143c600640 1 -- 172.21.15.26:0/3280875553 >> 172.21.15.26:0/3280875553 conn(0x7f143409b9e0 msgr2=0x7f143409bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:44.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.915+0000 7f958be00640 1 Processor -- start 2024-04-26T21:08:44.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.915+0000 7f958be00640 1 -- start start 2024-04-26T21:08:44.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.916+0000 7f8e4ea00640 1 -- 172.21.15.26:0/121753664 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 757773773 0 0) 0x7f8e500b4900 con 0x7f8e500d3dd0 2024-04-26T21:08:44.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.916+0000 7f28d6c00640 1 Processor -- start 2024-04-26T21:08:44.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.916+0000 7f8e55200640 1 -- 172.21.15.26:0/121753664 >> v1:172.21.15.86:6789/0 conn(0x7f8e500d3dd0 legacy=0x7f8e500b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.916+0000 7f8e55200640 1 -- 172.21.15.26:0/121753664 >> v1:172.21.15.26:6789/0 conn(0x7f8e500d39f0 legacy=0x7f8e5015f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.916+0000 7f8e55200640 1 -- 172.21.15.26:0/121753664 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e500b4ec0 con 0x7f8e5015f710 2024-04-26T21:08:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.916+0000 7fed86000640 1 -- 172.21.15.26:0/314025201 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed7c03d690 con 0x7fed90163730 2024-04-26T21:08:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.916+0000 7f28d6c00640 1 -- start start 2024-04-26T21:08:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.915+0000 7fe72ca00640 1 -- 172.21.15.26:0/1605825689 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7280d39f0 msgr2=0x7fe728160ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.915+0000 7fe72de00640 1 --2- 172.21.15.26:0/1605825689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe72815f000 0x7fe7281f3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.917+0000 7fe72ca00640 1 --2- 172.21.15.26:0/1605825689 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7280d39f0 0x7fe728160ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.917+0000 7fe72ca00640 1 -- 172.21.15.26:0/1605825689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe72815f000 msgr2=0x7fe7281f3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.917+0000 7fe72ca00640 1 --2- 172.21.15.26:0/1605825689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe72815f000 0x7fe7281f3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.917+0000 7fe72ca00640 1 -- 172.21.15.26:0/1605825689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72815fa10 con 0x7fe7280d3dd0 2024-04-26T21:08:44.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.917+0000 7fedf5800640 1 Processor -- start 2024-04-26T21:08:44.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.917+0000 7fe72de00640 1 --2- 172.21.15.26:0/1605825689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe72815f000 0x7fe7281f3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:44.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.917+0000 7f28d6c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28d00b4470 0x7f28d00ce350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.917+0000 7f28d6c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28d00cfd30 0x7f28d015aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.917+0000 7f28d6c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d00d4000 0x7f28d0158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.917+0000 7f143c600640 1 -- 172.21.15.26:0/3280875553 shutdown_connections 2024-04-26T21:08:44.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.917+0000 7f28d6c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f28d00d0b90 con 0x7f28d00b4470 2024-04-26T21:08:44.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.918+0000 7f28d6c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28d00d08b0 con 0x7f28d00d4000 2024-04-26T21:08:44.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.918+0000 7f28d6c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f28d00b5bb0 con 0x7f28d00cfd30 2024-04-26T21:08:44.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.917+0000 7f958be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95840b63f0 0x7f95840d5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.917+0000 7f958be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f95840b67d0 0x7f95840d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.917+0000 7f8e55200640 1 --2- 172.21.15.26:0/121753664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e5015f710 0x7f8e500d5cc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009d90 tx=0x7f8e4802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27929fccf828c426 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.918+0000 7f28cfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28d00b4470 0x7f28d00ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.918+0000 7f28cfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28d00b4470 0x7f28d00ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34594/0 (socket says 172.21.15.26:34594) 2024-04-26T21:08:44.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.918+0000 7f28cfe00640 1 -- 172.21.15.26:0/1303997293 learned_addr learned my addr 172.21.15.26:0/1303997293 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:44.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.918+0000 7f143c600640 1 -- 172.21.15.26:0/3280875553 wait complete. 2024-04-26T21:08:44.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.918+0000 7f8e4ea00640 1 -- 172.21.15.26:0/121753664 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48007740 con 0x7f8e5015f710 2024-04-26T21:08:44.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.918+0000 7f8e4ea00640 1 -- 172.21.15.26:0/121753664 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e480078e0 con 0x7f8e5015f710 2024-04-26T21:08:44.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.918+0000 7f8e4ea00640 1 -- 172.21.15.26:0/121753664 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48007cb0 con 0x7f8e5015f710 2024-04-26T21:08:44.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.918+0000 7fe72ca00640 1 --2- 172.21.15.26:0/1605825689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7280d3dd0 0x7fe7281f9f40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe71400d5f0 tx=0x7fe71400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7fe725600640 1 -- 172.21.15.26:0/1605825689 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7140125f0 con 0x7fe7280d3dd0 2024-04-26T21:08:44.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7fb6fde00640 1 Processor -- start 2024-04-26T21:08:44.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7fed86000640 1 --2- 172.21.15.26:0/314025201 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fed7006caf0 0x7fed7006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7fed86000640 1 -- 172.21.15.26:0/314025201 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fed7c0aba80 con 0x7fed90163730 2024-04-26T21:08:44.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7fe725600640 1 -- 172.21.15.26:0/1605825689 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe714019070 con 0x7fe7280d3dd0 2024-04-26T21:08:44.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7fe725600640 1 -- 172.21.15.26:0/1605825689 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7140129b0 con 0x7fe7280d3dd0 2024-04-26T21:08:44.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.918+0000 7fedf5800640 1 -- start start 2024-04-26T21:08:44.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7f8e57200640 1 -- 172.21.15.26:0/121753664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e5015f710 msgr2=0x7f8e500d5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7f8e57200640 1 --2- 172.21.15.26:0/121753664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e5015f710 0x7f8e500d5cc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009d90 tx=0x7f8e4802a6f0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7fed8de00640 1 --2- 172.21.15.26:0/314025201 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fed7006caf0 0x7fed7006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7fed86000640 1 --2- 172.21.15.26:0/314025201 >> [v2:172.21.15.26:6808/2924039897,v1:172.21.15.26:6811/2924039897] conn(0x7fed70074d10 0x7fed70077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7fed86000640 1 -- 172.21.15.26:0/314025201 --> [v2:172.21.15.26:6808/2924039897,v1:172.21.15.26:6811/2924039897] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed7c0067f0 con 0x7fed70074d10 2024-04-26T21:08:44.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7fe72fe00640 1 -- 172.21.15.26:0/1605825689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe72815fcf0 con 0x7fe7280d3dd0 2024-04-26T21:08:44.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.920+0000 7fe72fe00640 1 -- 172.21.15.26:0/1605825689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7281602b0 con 0x7fe7280d3dd0 2024-04-26T21:08:44.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.920+0000 7fedf5800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedf0070120 0x7fedf014fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7fed86000640 1 -- 172.21.15.26:0/314025201 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fed7c0ade60 con 0x7fed90163730 2024-04-26T21:08:44.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.920+0000 7fed8e800640 1 --2- 172.21.15.26:0/314025201 >> [v2:172.21.15.26:6808/2924039897,v1:172.21.15.26:6811/2924039897] conn(0x7fed70074d10 0x7fed70077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.920+0000 7fedf5800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fedf0073d30 0x7fedf01f0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.920+0000 7fedf5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf00b3d10 0x7fedf0159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.921+0000 7fedf5800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fedf00b37f0 con 0x7fedf0070120 2024-04-26T21:08:44.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.921+0000 7fedf5800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fedf00b3670 con 0x7fedf00b3d10 2024-04-26T21:08:44.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.921+0000 7fedef400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fedf0073d30 0x7fedf01f0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.921+0000 7fedef400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fedf0073d30 0x7fedf01f0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.26:45950/0 (socket says 172.21.15.26:45950) 2024-04-26T21:08:44.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.921+0000 7fedef400640 1 -- 172.21.15.26:0/667080570 learned_addr learned my addr 172.21.15.26:0/667080570 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:44.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.921+0000 7fe72fe00640 1 -- 172.21.15.26:0/1605825689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe72806a700 con 0x7fe7280d3dd0 2024-04-26T21:08:44.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7f958be00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95840c8970 con 0x7f95840d1fb0 2024-04-26T21:08:44.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7f958be00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f95840d1290 con 0x7f95840b67d0 2024-04-26T21:08:44.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7f958be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f95840d0960 con 0x7f95840b63f0 2024-04-26T21:08:44.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7f9588a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f95840b67d0 0x7f95840d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7f9589e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f95840d1fb0 0x7f95840d30a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.26:33994/0 (socket says 172.21.15.26:33994) 2024-04-26T21:08:44.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7f9589e00640 1 -- 172.21.15.26:0/2624888433 learned_addr learned my addr 172.21.15.26:0/2624888433 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:44.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.921+0000 7f9583e00640 1 -- 172.21.15.26:0/2624888433 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 68291576 0 0) 0x7f95840c8970 con 0x7f95840d1fb0 2024-04-26T21:08:44.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.921+0000 7f9583e00640 1 -- 172.21.15.26:0/2624888433 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f956c003520 con 0x7f95840d1fb0 2024-04-26T21:08:44.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.921+0000 7f8e57200640 1 -- 172.21.15.26:0/121753664 shutdown_connections 2024-04-26T21:08:44.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.921+0000 7f8e57200640 1 --2- 172.21.15.26:0/121753664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e5015f710 0x7f8e500d5cc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.921+0000 7f8e57200640 1 -- 172.21.15.26:0/121753664 >> 172.21.15.26:0/121753664 conn(0x7f8e500d51a0 msgr2=0x7f8e500d55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:44.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.921+0000 7fedf5800640 1 -- 172.21.15.26:0/667080570 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fedf0072c90 con 0x7fedf0073d30 2024-04-26T21:08:44.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.921+0000 7f9583e00640 1 -- 172.21.15.26:0/2624888433 <== mon.1 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1232442251 0 0) 0x7f956c003520 con 0x7f95840d1fb0 2024-04-26T21:08:44.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.921+0000 7f8e57200640 1 -- 172.21.15.26:0/121753664 shutdown_connections 2024-04-26T21:08:44.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7fed8e800640 1 --2- 172.21.15.26:0/314025201 >> [v2:172.21.15.26:6808/2924039897,v1:172.21.15.26:6811/2924039897] conn(0x7fed70074d10 0x7fed70077130 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7f9589400640 1 --2- 172.21.15.26:0/2624888433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95840b63f0 0x7f95840d5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7fedf4e00640 1 --2- 172.21.15.26:0/667080570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf00b3d10 0x7fedf0159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7fedefe00640 1 --2- 172.21.15.26:0/667080570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedf0070120 0x7fedf014fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7f9583e00640 1 -- 172.21.15.26:0/2624888433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95840b63f0 msgr2=0x7f95840d5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:44.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7f9583e00640 1 --2- 172.21.15.26:0/2624888433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95840b63f0 0x7f95840d5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7f9583e00640 1 -- 172.21.15.26:0/2624888433 >> v2:172.21.15.86:3300/0 conn(0x7f95840b67d0 msgr2=0x7f95840d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:44.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7f9583e00640 1 --2- 172.21.15.26:0/2624888433 >> v2:172.21.15.86:3300/0 conn(0x7f95840b67d0 0x7f95840d1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7f9583e00640 1 -- 172.21.15.26:0/2624888433 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95840d0c40 con 0x7f95840d1fb0 2024-04-26T21:08:44.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7f9583e00640 1 -- 172.21.15.26:0/2624888433 <== mon.1 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f957c0019f0 con 0x7f95840d1fb0 2024-04-26T21:08:44.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7f9583e00640 1 -- 172.21.15.26:0/2624888433 >> v1:172.21.15.86:6789/0 conn(0x7f95840d1fb0 legacy=0x7f95840d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7f9583e00640 1 --2- 172.21.15.26:0/2624888433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f956c0039b0 0x7f956c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7f9583e00640 1 --2- 172.21.15.26:0/2624888433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956c004800 0x7f956c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7f9583e00640 1 --2- 172.21.15.26:0/2624888433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f956c0051c0 0x7f956c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7f9583e00640 1 -- 172.21.15.26:0/2624888433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f95840d1290 con 0x7f956c004800 2024-04-26T21:08:44.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7f9583e00640 1 -- 172.21.15.26:0/2624888433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95840d0960 con 0x7f956c0051c0 2024-04-26T21:08:44.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7f9583e00640 1 -- 172.21.15.26:0/2624888433 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f956c00c0b0 con 0x7f956c0039b0 2024-04-26T21:08:44.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.920+0000 7fb6fde00640 1 -- start start 2024-04-26T21:08:44.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7fb6fde00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb6f8160670 0x7fb6f80d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7fb6fde00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6f80d2120 0x7fb6f80b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7fb6fde00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6f80d0630 con 0x7fb6f815f4e0 2024-04-26T21:08:44.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7fb6fde00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb6f80b4b30 con 0x7fb6f8160670 2024-04-26T21:08:44.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7fb6fde00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f80d0910 con 0x7fb6f80d2120 2024-04-26T21:08:44.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7f28cf400640 1 --2- 172.21.15.26:0/1303997293 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28d00cfd30 0x7f28d015aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7f28cfe00640 1 -- 172.21.15.26:0/1303997293 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28d00cfd30 msgr2=0x7f28d015aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7f28cfe00640 1 --2- 172.21.15.26:0/1303997293 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28d00cfd30 0x7f28d015aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7f28cfe00640 1 -- 172.21.15.26:0/1303997293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d00d4000 msgr2=0x7f28d0158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7f28cfe00640 1 --2- 172.21.15.26:0/1303997293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d00d4000 0x7f28d0158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.919+0000 7f28cfe00640 1 -- 172.21.15.26:0/1303997293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d0158cc0 con 0x7f28d00b4470 2024-04-26T21:08:44.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.920+0000 7f28cfe00640 1 --2- 172.21.15.26:0/1303997293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28d00b4470 0x7f28d00ce350 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f28c800efc0 tx=0x7f28c800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.920+0000 7f28c7e00640 1 -- 172.21.15.26:0/1303997293 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c800cda0 con 0x7f28d00b4470 2024-04-26T21:08:44.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.920+0000 7f28c7e00640 1 -- 172.21.15.26:0/1303997293 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28c8019070 con 0x7f28d00b4470 2024-04-26T21:08:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.920+0000 7f28c7e00640 1 -- 172.21.15.26:0/1303997293 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c800cda0 con 0x7f28d00b4470 2024-04-26T21:08:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.920+0000 7f28d6c00640 1 -- 172.21.15.26:0/1303997293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d015b220 con 0x7f28d00b4470 2024-04-26T21:08:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.921+0000 7f28d6c00640 1 -- 172.21.15.26:0/1303997293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d015b7e0 con 0x7f28d00b4470 2024-04-26T21:08:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.921+0000 7f143c600640 1 Processor -- start 2024-04-26T21:08:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.923+0000 7f143c600640 1 -- start start 2024-04-26T21:08:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.923+0000 7f9589400640 1 --2- 172.21.15.26:0/2624888433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f956c0039b0 0x7f956c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.923+0000 7f9589e00640 1 --2- 172.21.15.26:0/2624888433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f956c0051c0 0x7f956c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.923+0000 7fb6f6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb6f8160670 0x7fb6f80d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.923+0000 7fb6f6a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb6f815f4e0 0x7fb6f80cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.26:33996/0 (socket says 172.21.15.26:33996) 2024-04-26T21:08:44.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.923+0000 7fb6f6a00640 1 -- 172.21.15.26:0/2305653230 learned_addr learned my addr 172.21.15.26:0/2305653230 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:44.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.923+0000 7fb6f7400640 1 --2- 172.21.15.26:0/2305653230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6f80d2120 0x7fb6f80b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7fed8de00640 1 --2- 172.21.15.26:0/314025201 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fed7006caf0 0x7fed7006efb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fed800094a0 tx=0x7fed80009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.922+0000 7f8e57200640 1 -- 172.21.15.26:0/121753664 wait complete. 2024-04-26T21:08:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.924+0000 7fedef400640 1 -- 172.21.15.26:0/667080570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf00b3d10 msgr2=0x7fedf0159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.924+0000 7fedef400640 1 --2- 172.21.15.26:0/667080570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fedf00b3d10 0x7fedf0159030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.924+0000 7fedef400640 1 -- 172.21.15.26:0/667080570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedf0070120 msgr2=0x7fedf014fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.924+0000 7fedef400640 1 --2- 172.21.15.26:0/667080570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedf0070120 0x7fedf014fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.924+0000 7fedef400640 1 -- 172.21.15.26:0/667080570 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf0150d00 con 0x7fedf0073d30 2024-04-26T21:08:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.924+0000 7fb6f6000640 1 -- 172.21.15.26:0/2305653230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6f80d2120 msgr2=0x7fb6f80b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.924+0000 7fb6f6000640 1 --2- 172.21.15.26:0/2305653230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6f80d2120 0x7fb6f80b5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.924+0000 7fb6f6000640 1 -- 172.21.15.26:0/2305653230 >> v1:172.21.15.86:6789/0 conn(0x7fb6f815f4e0 legacy=0x7fb6f80cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.924+0000 7fb6f6000640 1 -- 172.21.15.26:0/2305653230 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f80d0bf0 con 0x7fb6f8160670 2024-04-26T21:08:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.924+0000 7f9588a00640 1 --2- 172.21.15.26:0/2624888433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956c004800 0x7f956c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.924+0000 7fb6f7400640 1 --2- 172.21.15.26:0/2305653230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6f80d2120 0x7fb6f80b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:08:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.924+0000 7f9589400640 1 -- 172.21.15.26:0/2624888433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f956c0051c0 msgr2=0x7f956c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.924+0000 7f9589400640 1 --2- 172.21.15.26:0/2624888433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f956c0051c0 0x7f956c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.924+0000 7f9589400640 1 -- 172.21.15.26:0/2624888433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956c004800 msgr2=0x7f956c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.924+0000 7f9589400640 1 --2- 172.21.15.26:0/2624888433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956c004800 0x7f956c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.924+0000 7f9589400640 1 -- 172.21.15.26:0/2624888433 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95840d0c40 con 0x7f956c0039b0 2024-04-26T21:08:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.924+0000 7f9588a00640 1 --2- 172.21.15.26:0/2624888433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956c004800 0x7f956c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.924+0000 7fed86000640 1 -- 172.21.15.26:0/314025201 <== osd.0 v2:172.21.15.26:6808/2924039897 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed7c0067f0 con 0x7fed70074d10 2024-04-26T21:08:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.924+0000 7fe725600640 1 -- 172.21.15.26:0/1605825689 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe714012b50 con 0x7fe7280d3dd0 2024-04-26T21:08:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.924+0000 7fedef400640 1 --2- 172.21.15.26:0/667080570 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fedf0073d30 0x7fedf01f0b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7feddc00d800 tx=0x7feddc00dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.924+0000 7f8e57200640 1 Processor -- start 2024-04-26T21:08:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.925+0000 7f28d6c00640 1 -- 172.21.15.26:0/1303997293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f28d015be30 con 0x7f28d00b4470 2024-04-26T21:08:44.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.925+0000 7fede7e00640 1 -- 172.21.15.26:0/667080570 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc00b6c0 con 0x7fedf0073d30 2024-04-26T21:08:44.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.925+0000 7fede7e00640 1 -- 172.21.15.26:0/667080570 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feddc019070 con 0x7fedf0073d30 2024-04-26T21:08:44.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.925+0000 7fede7e00640 1 -- 172.21.15.26:0/667080570 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc00ba80 con 0x7fedf0073d30 2024-04-26T21:08:44.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.925+0000 7fedf5800640 1 -- 172.21.15.26:0/667080570 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf0156820 con 0x7fedf0073d30 2024-04-26T21:08:44.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.925+0000 7fedf5800640 1 -- 172.21.15.26:0/667080570 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf0156de0 con 0x7fedf0073d30 2024-04-26T21:08:44.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.925+0000 7f143c600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14340cf710 0x7f14340c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.926+0000 7f143c600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f143415f130 0x7f14341f8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.926+0000 7f143c600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14341f8d70 0x7f1434071320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.926+0000 7f143c600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f14340afc20 con 0x7f14340cf710 2024-04-26T21:08:44.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.926+0000 7f143c600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14340cde20 con 0x7f143415f130 2024-04-26T21:08:44.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.926+0000 7f143c600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1434158c30 con 0x7f14341f8d70 2024-04-26T21:08:44.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.926+0000 7fe725600640 1 --2- 172.21.15.26:0/1605825689 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe70806c940 0x7fe70806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.926+0000 7fe725600640 1 -- 172.21.15.26:0/1605825689 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe71408c800 con 0x7fe7280d3dd0 2024-04-26T21:08:44.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.926+0000 7fe72d400640 1 --2- 172.21.15.26:0/1605825689 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe70806c940 0x7fe70806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.926+0000 7fe725600640 1 --2- 172.21.15.26:0/1605825689 >> [v2:172.21.15.26:6800/398686833,v1:172.21.15.26:6801/398686833] conn(0x7fe708074b60 0x7fe708076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.926+0000 7fe725600640 1 -- 172.21.15.26:0/1605825689 --> [v2:172.21.15.26:6800/398686833,v1:172.21.15.26:6801/398686833] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe71400cfb0 con 0x7fe708074b60 2024-04-26T21:08:44.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.926+0000 7fe725600640 1 -- 172.21.15.26:0/1605825689 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe71408cce0 con 0x7fe7280d3dd0 2024-04-26T21:08:44.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.927+0000 7fedf5800640 1 -- 172.21.15.26:0/667080570 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fedf006a700 con 0x7fedf0073d30 2024-04-26T21:08:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.927+0000 7fe72de00640 1 --2- 172.21.15.26:0/1605825689 >> [v2:172.21.15.26:6800/398686833,v1:172.21.15.26:6801/398686833] conn(0x7fe708074b60 0x7fe708076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.927+0000 7f143a600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14341f8d70 0x7f1434071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.927+0000 7f9589400640 1 --2- 172.21.15.26:0/2624888433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f956c0039b0 0x7f956c003dd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f95840d2ca0 tx=0x7f956c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3227ab654c2d3296 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.927+0000 7fb6f6000640 1 --2- 172.21.15.26:0/2305653230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6f8160670 0x7fb6f80d1be0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb6e40097a0 tx=0x7fb6e402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ab5e727eddef8ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.927+0000 7f143a600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14341f8d70 0x7f1434071320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.26:45968/0 (socket says 172.21.15.26:45968) 2024-04-26T21:08:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.927+0000 7f9583e00640 1 -- 172.21.15.26:0/2624888433 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570037070 con 0x7f956c0039b0 2024-04-26T21:08:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.928+0000 7f28c7e00640 1 -- 172.21.15.26:0/1303997293 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28c8005d10 con 0x7f28d00b4470 2024-04-26T21:08:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.927+0000 7f143a600640 1 -- 172.21.15.26:0/2612816864 learned_addr learned my addr 172.21.15.26:0/2612816864 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.928+0000 7f9583e00640 1 -- 172.21.15.26:0/2624888433 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f957002aa20 con 0x7f956c0039b0 2024-04-26T21:08:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.928+0000 7fb6f5600640 1 -- 172.21.15.26:0/2305653230 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4037070 con 0x7fb6f8160670 2024-04-26T21:08:44.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.928+0000 7f9583e00640 1 -- 172.21.15.26:0/2624888433 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957000a4b0 con 0x7f956c0039b0 2024-04-26T21:08:44.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.928+0000 7fe72d400640 1 --2- 172.21.15.26:0/1605825689 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe70806c940 0x7fe70806ee00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe71c02a1c0 tx=0x7fe71c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.927+0000 7f8e57200640 1 -- start start 2024-04-26T21:08:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.928+0000 7fb6f5600640 1 -- 172.21.15.26:0/2305653230 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e4032440 con 0x7fb6f8160670 2024-04-26T21:08:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.928+0000 7fb6f5600640 1 -- 172.21.15.26:0/2305653230 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4032810 con 0x7fb6f8160670 2024-04-26T21:08:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.929+0000 7fb6fde00640 1 -- 172.21.15.26:0/2305653230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6f8160670 msgr2=0x7fb6f80d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.929+0000 7fb6fde00640 1 --2- 172.21.15.26:0/2305653230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6f8160670 0x7fb6f80d1be0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb6e40097a0 tx=0x7fb6e402a180 comp rx=0 tx=0).stop 2024-04-26T21:08:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.929+0000 7f958be00640 1 -- 172.21.15.26:0/2624888433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f956c0039b0 msgr2=0x7f956c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.929+0000 7f958be00640 1 --2- 172.21.15.26:0/2624888433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f956c0039b0 0x7f956c003dd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f95840d2ca0 tx=0x7f956c005830 comp rx=0 tx=0).stop 2024-04-26T21:08:44.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.929+0000 7f143a600640 1 -- 172.21.15.26:0/2612816864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f143415f130 msgr2=0x7f14341f8830 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:08:44.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.929+0000 7f143a600640 1 --2- 172.21.15.26:0/2612816864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f143415f130 0x7f14341f8830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.929+0000 7fe72de00640 1 --2- 172.21.15.26:0/1605825689 >> [v2:172.21.15.26:6800/398686833,v1:172.21.15.26:6801/398686833] conn(0x7fe708074b60 0x7fe708076f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.929+0000 7f143a600640 1 -- 172.21.15.26:0/2612816864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14340cf710 msgr2=0x7f14340c9f60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:08:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.929+0000 7f143a600640 1 --2- 172.21.15.26:0/2612816864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14340cf710 0x7f14340c9f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.929+0000 7f143a600640 1 -- 172.21.15.26:0/2612816864 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14340ca4a0 con 0x7f14341f8d70 2024-04-26T21:08:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.929+0000 7fb6fde00640 1 -- 172.21.15.26:0/2305653230 shutdown_connections 2024-04-26T21:08:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.929+0000 7fb6fde00640 1 --2- 172.21.15.26:0/2305653230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6f80d2120 0x7fb6f80b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.929+0000 7fb6fde00640 1 --2- 172.21.15.26:0/2305653230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6f8160670 0x7fb6f80d1be0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.929+0000 7fb6fde00640 1 -- 172.21.15.26:0/2305653230 >> 172.21.15.26:0/2305653230 conn(0x7fb6f809e4b0 msgr2=0x7fb6f80d51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.929+0000 7f9589e00640 1 -- 172.21.15.26:0/2624888433 reap_dead start 2024-04-26T21:08:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.929+0000 7f958be00640 1 -- 172.21.15.26:0/2624888433 shutdown_connections 2024-04-26T21:08:44.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.929+0000 7f958be00640 1 -- 172.21.15.26:0/2624888433 >> 172.21.15.26:0/2624888433 conn(0x7f95840d51a0 msgr2=0x7f95840d55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:44.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.929+0000 7fb6fde00640 1 -- 172.21.15.26:0/2305653230 shutdown_connections 2024-04-26T21:08:44.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.929+0000 7fb6fde00640 1 -- 172.21.15.26:0/2305653230 wait complete. 2024-04-26T21:08:44.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.929+0000 7f143a600640 1 --2- 172.21.15.26:0/2612816864 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14341f8d70 0x7f1434071320 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f142c00c790 tx=0x7f142c00cc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.929+0000 7f958be00640 1 -- 172.21.15.26:0/2624888433 shutdown_connections 2024-04-26T21:08:44.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.930+0000 7f1431600640 1 -- 172.21.15.26:0/2612816864 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c003b70 con 0x7f14341f8d70 2024-04-26T21:08:44.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.930+0000 7f1431600640 1 -- 172.21.15.26:0/2612816864 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f142c003d10 con 0x7f14341f8d70 2024-04-26T21:08:44.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.930+0000 7f1431600640 1 -- 172.21.15.26:0/2612816864 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c00a9f0 con 0x7f14341f8d70 2024-04-26T21:08:44.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.931+0000 7fe725600640 1 -- 172.21.15.26:0/1605825689 <== osd.1 v2:172.21.15.26:6800/398686833 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe71400cfb0 con 0x7fe708074b60 2024-04-26T21:08:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.930+0000 7f28c7e00640 1 --2- 172.21.15.26:0/1303997293 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f28b006caf0 0x7f28b006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.930+0000 7f28c7e00640 1 -- 172.21.15.26:0/1303997293 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f28c808bca0 con 0x7f28d00b4470 2024-04-26T21:08:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.930+0000 7f28c7e00640 1 --2- 172.21.15.26:0/1303997293 >> [v2:172.21.15.26:6824/2316564464,v1:172.21.15.26:6825/2316564464] conn(0x7f28b0074cf0 0x7f28b0077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.930+0000 7f28c7e00640 1 -- 172.21.15.26:0/1303997293 --> [v2:172.21.15.26:6824/2316564464,v1:172.21.15.26:6825/2316564464] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f28b00777c0 con 0x7f28b0074cf0 2024-04-26T21:08:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.930+0000 7f28c7e00640 1 -- 172.21.15.26:0/1303997293 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f28c808e3a0 con 0x7f28d00b4470 2024-04-26T21:08:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.931+0000 7f28d4c00640 1 --2- 172.21.15.26:0/1303997293 >> [v2:172.21.15.26:6824/2316564464,v1:172.21.15.26:6825/2316564464] conn(0x7f28b0074cf0 0x7f28b0077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.930+0000 7f143c600640 1 -- 172.21.15.26:0/2612816864 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14340ca750 con 0x7f14341f8d70 2024-04-26T21:08:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.930+0000 7f143c600640 1 -- 172.21.15.26:0/2612816864 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f14340cad10 con 0x7f14341f8d70 2024-04-26T21:08:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.929+0000 7f958be00640 1 -- 172.21.15.26:0/2624888433 wait complete. 2024-04-26T21:08:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.931+0000 7f958be00640 1 Processor -- start 2024-04-26T21:08:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.931+0000 7fb6fde00640 1 Processor -- start 2024-04-26T21:08:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.932+0000 7fb6fde00640 1 -- start start 2024-04-26T21:08:44.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.932+0000 7f28d4c00640 1 --2- 172.21.15.26:0/1303997293 >> [v2:172.21.15.26:6824/2316564464,v1:172.21.15.26:6825/2316564464] conn(0x7f28b0074cf0 0x7f28b0077110 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.932+0000 7f28cf400640 1 --2- 172.21.15.26:0/1303997293 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f28b006caf0 0x7f28b006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.932+0000 7f958be00640 1 -- start start 2024-04-26T21:08:44.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.931+0000 7fede7e00640 1 -- 172.21.15.26:0/667080570 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feddc00bc20 con 0x7fedf0073d30 2024-04-26T21:08:44.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.932+0000 7fede7e00640 1 --2- 172.21.15.26:0/667080570 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fedd006c8f0 0x7fedd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.932+0000 7fede7e00640 1 -- 172.21.15.26:0/667080570 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7feddc08ca40 con 0x7fedf0073d30 2024-04-26T21:08:44.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.932+0000 7fede7e00640 1 --2- 172.21.15.26:0/667080570 >> [v2:172.21.15.86:6816/1450967413,v1:172.21.15.86:6817/1450967413] conn(0x7fedd0074b10 0x7fedd0076f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.932+0000 7fede7e00640 1 -- 172.21.15.26:0/667080570 --> [v2:172.21.15.86:6816/1450967413,v1:172.21.15.86:6817/1450967413] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fedd00775e0 con 0x7fedd0074b10 2024-04-26T21:08:44.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.932+0000 7fede7e00640 1 -- 172.21.15.26:0/667080570 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7feddc08ce30 con 0x7fedf0073d30 2024-04-26T21:08:44.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.932+0000 7f958be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95840ce5c0 0x7f95840cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.933+0000 7fedefe00640 1 --2- 172.21.15.26:0/667080570 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fedd006c8f0 0x7fedd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.933+0000 7f958be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95840cddb0 0x7f95841644a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.933+0000 7fb6fde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6f80d2120 0x7fb6f80b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.933+0000 7fb6fde00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6f815f4e0 0x7fb6f8149390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.933+0000 7fb6fde00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6f8160670 0x7fb6f80b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.933+0000 7fb6fde00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f815f8c0 con 0x7fb6f8160670 2024-04-26T21:08:44.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.933+0000 7fb6fde00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f80d11b0 con 0x7fb6f80d2120 2024-04-26T21:08:44.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.933+0000 7fb6fde00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f80b6310 con 0x7fb6f815f4e0 2024-04-26T21:08:44.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.934+0000 7f958be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95840cc7b0 0x7f95841649e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.934+0000 7f958be00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f95840b93f0 con 0x7f95840cc7b0 2024-04-26T21:08:44.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.934+0000 7f958be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95840b9270 con 0x7f95840ce5c0 2024-04-26T21:08:44.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.934+0000 7f28c7e00640 1 -- 172.21.15.26:0/1303997293 <== osd.2 v2:172.21.15.26:6824/2316564464 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f28b00777c0 con 0x7f28b0074cf0 2024-04-26T21:08:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.934+0000 7f958be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f95840b9570 con 0x7f95840cddb0 2024-04-26T21:08:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.934+0000 7f9589400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95840ce5c0 0x7f95840cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.934+0000 7fb6f6000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6f815f4e0 0x7fb6f8149390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.934+0000 7f9589400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95840ce5c0 0x7f95840cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.26:41766/0 (socket says 172.21.15.26:41766) 2024-04-26T21:08:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.934+0000 7f9589400640 1 -- 172.21.15.26:0/1833400496 learned_addr learned my addr 172.21.15.26:0/1833400496 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.934+0000 7f9588a00640 1 --2- 172.21.15.26:0/1833400496 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95840cddb0 0x7f95841644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.934+0000 7f9589e00640 1 --2- 172.21.15.26:0/1833400496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95840cc7b0 0x7f95841649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.931+0000 7f143c600640 1 -- 172.21.15.26:0/2612816864 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f14340caf50 con 0x7f14341f8d70 2024-04-26T21:08:44.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.934+0000 7fedefe00640 1 --2- 172.21.15.26:0/667080570 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fedd006c8f0 0x7fedd006edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fede0045000 tx=0x7fede003f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.935+0000 7f28cf400640 1 --2- 172.21.15.26:0/1303997293 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f28b006caf0 0x7f28b006efb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f28c0034000 tx=0x7f28c002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.935+0000 7fb6f6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6f80d2120 0x7fb6f80b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.935+0000 7fb6f6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6f80d2120 0x7fb6f80b37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.26:41776/0 (socket says 172.21.15.26:41776) 2024-04-26T21:08:44.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.935+0000 7fb6f6a00640 1 -- 172.21.15.26:0/2508509653 learned_addr learned my addr 172.21.15.26:0/2508509653 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:44.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.935+0000 7fb6f7400640 1 --2- 172.21.15.26:0/2508509653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6f8160670 0x7fb6f80b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.935+0000 7f8e57200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e500d39f0 0x7f8e5015e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.935+0000 7f8e57200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e500d3dd0 0x7f8e50160b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.935+0000 7f8e57200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e5015f710 0x7f8e500ba420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.935+0000 7f8e57200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8e500d0b90 con 0x7f8e500d3dd0 2024-04-26T21:08:44.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.935+0000 7f8e57200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e500d0a10 con 0x7f8e500d39f0 2024-04-26T21:08:44.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.935+0000 7f8e57200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8e500b5af0 con 0x7f8e5015f710 2024-04-26T21:08:44.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.935+0000 7fedf4e00640 1 --2- 172.21.15.26:0/667080570 >> [v2:172.21.15.86:6816/1450967413,v1:172.21.15.86:6817/1450967413] conn(0x7fedd0074b10 0x7fedd0076f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.936+0000 7f1431600640 1 -- 172.21.15.26:0/2612816864 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f142c00ab90 con 0x7f14341f8d70 2024-04-26T21:08:44.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.936+0000 7f8e4f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e500d3dd0 0x7f8e50160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.936+0000 7f8e4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e500d39f0 0x7f8e5015e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.936+0000 7f8e4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e500d39f0 0x7f8e5015e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.26:41782/0 (socket says 172.21.15.26:41782) 2024-04-26T21:08:44.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.936+0000 7f8e4fe00640 1 -- 172.21.15.26:0/2107785028 learned_addr learned my addr 172.21.15.26:0/2107785028 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:44.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.936+0000 7f9589400640 1 -- 172.21.15.26:0/1833400496 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95840cddb0 msgr2=0x7f95841644a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.936+0000 7f9589400640 1 --2- 172.21.15.26:0/1833400496 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95840cddb0 0x7f95841644a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.936+0000 7f9589400640 1 -- 172.21.15.26:0/1833400496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95840cc7b0 msgr2=0x7f95841649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.936+0000 7f9589400640 1 --2- 172.21.15.26:0/1833400496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95840cc7b0 0x7f95841649e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.936+0000 7f9589400640 1 -- 172.21.15.26:0/1833400496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95840ccd10 con 0x7f95840ce5c0 2024-04-26T21:08:44.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.936+0000 7f8e55200640 1 --2- 172.21.15.26:0/2107785028 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e5015f710 0x7f8e500ba420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.937+0000 7f1431600640 1 --2- 172.21.15.26:0/2612816864 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f141406c8f0 0x7f141406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.937+0000 7f8e4f400640 1 -- 172.21.15.26:0/2107785028 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e5015f710 msgr2=0x7f8e500ba420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.937+0000 7f8e4f400640 1 --2- 172.21.15.26:0/2107785028 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e5015f710 0x7f8e500ba420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.937+0000 7f8e4f400640 1 -- 172.21.15.26:0/2107785028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e500d39f0 msgr2=0x7f8e5015e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.937+0000 7f8e4f400640 1 --2- 172.21.15.26:0/2107785028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e500d39f0 0x7f8e5015e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.936+0000 7fb6f7400640 1 -- 172.21.15.26:0/2508509653 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6f815f4e0 msgr2=0x7fb6f8149390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.936+0000 7fb6f7400640 1 --2- 172.21.15.26:0/2508509653 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6f815f4e0 0x7fb6f8149390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.936+0000 7fb6f7400640 1 -- 172.21.15.26:0/2508509653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6f80d2120 msgr2=0x7fb6f80b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.936+0000 7fb6f7400640 1 --2- 172.21.15.26:0/2508509653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6f80d2120 0x7fb6f80b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.936+0000 7fb6f7400640 1 -- 172.21.15.26:0/2508509653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f80ced00 con 0x7fb6f8160670 2024-04-26T21:08:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.937+0000 7f1431600640 1 -- 172.21.15.26:0/2612816864 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f142c08c5d0 con 0x7f14341f8d70 2024-04-26T21:08:44.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.937+0000 7f8e4f400640 1 -- 172.21.15.26:0/2107785028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e500b7d20 con 0x7f8e500d3dd0 2024-04-26T21:08:44.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.937+0000 7f9589400640 1 --2- 172.21.15.26:0/1833400496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95840ce5c0 0x7f95840cc270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f957002a510 tx=0x7f9570004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.937+0000 7f1431600640 1 --2- 172.21.15.26:0/2612816864 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x7f1414074a90 0x7f1414076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.937+0000 7fb6f7400640 1 --2- 172.21.15.26:0/2508509653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6f8160670 0x7fb6f80b6e10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0007f20 tx=0x7fb6f000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.937+0000 7f8e4f400640 1 --2- 172.21.15.26:0/2107785028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e500d3dd0 0x7f8e50160b70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00c750 tx=0x7f8e3c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.937+0000 7f1431600640 1 -- 172.21.15.26:0/2612816864 --> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f142c01fec0 con 0x7f1414074a90 2024-04-26T21:08:44.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.937+0000 7f1431600640 1 -- 172.21.15.26:0/2612816864 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f142c08c980 con 0x7f14341f8d70 2024-04-26T21:08:44.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.937+0000 7f1439c00640 1 --2- 172.21.15.26:0/2612816864 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f141406c8f0 0x7f141406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.938+0000 7f9581600640 1 -- 172.21.15.26:0/1833400496 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570037050 con 0x7f95840ce5c0 2024-04-26T21:08:44.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.938+0000 7f9581600640 1 -- 172.21.15.26:0/1833400496 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9570043070 con 0x7f95840ce5c0 2024-04-26T21:08:44.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.938+0000 7f9581600640 1 -- 172.21.15.26:0/1833400496 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957003e690 con 0x7f95840ce5c0 2024-04-26T21:08:44.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.938+0000 7f8e47e00640 1 -- 172.21.15.26:0/2107785028 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c018070 con 0x7f8e500d3dd0 2024-04-26T21:08:44.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.938+0000 7f8e47e00640 1 -- 172.21.15.26:0/2107785028 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e3c0070d0 con 0x7f8e500d3dd0 2024-04-26T21:08:44.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.938+0000 7f8e47e00640 1 -- 172.21.15.26:0/2107785028 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c0074e0 con 0x7f8e500d3dd0 2024-04-26T21:08:44.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.938+0000 7fb6ef400640 1 -- 172.21.15.26:0/2508509653 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f001a070 con 0x7fb6f8160670 2024-04-26T21:08:44.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.938+0000 7fb6ef400640 1 -- 172.21.15.26:0/2508509653 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f0003b90 con 0x7fb6f8160670 2024-04-26T21:08:44.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.938+0000 7fb6ef400640 1 -- 172.21.15.26:0/2508509653 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0004030 con 0x7fb6f8160670 2024-04-26T21:08:44.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.938+0000 7f1439200640 1 --2- 172.21.15.26:0/2612816864 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x7f1414074a90 0x7f1414076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.938+0000 7f958be00640 1 -- 172.21.15.26:0/1833400496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95840ccff0 con 0x7f95840ce5c0 2024-04-26T21:08:44.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.939+0000 7f1439200640 1 --2- 172.21.15.26:0/2612816864 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x7f1414074a90 0x7f1414076eb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.939+0000 7f8e57200640 1 -- 172.21.15.26:0/2107785028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e501615a0 con 0x7f8e500d3dd0 2024-04-26T21:08:44.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.939+0000 7f8e57200640 1 -- 172.21.15.26:0/2107785028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e500b7fe0 con 0x7f8e500d3dd0 2024-04-26T21:08:44.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.939+0000 7fb6fde00640 1 -- 172.21.15.26:0/2508509653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f80cefe0 con 0x7fb6f8160670 2024-04-26T21:08:44.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.939+0000 7fb6fde00640 1 -- 172.21.15.26:0/2508509653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f80b3cf0 con 0x7fb6f8160670 2024-04-26T21:08:44.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.939+0000 7fedf4e00640 1 --2- 172.21.15.26:0/667080570 >> [v2:172.21.15.86:6816/1450967413,v1:172.21.15.86:6817/1450967413] conn(0x7fedd0074b10 0x7fedd0076f30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.938+0000 7f958be00640 1 -- 172.21.15.26:0/1833400496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95840cd5b0 con 0x7f95840ce5c0 2024-04-26T21:08:44.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.940+0000 7f958be00640 1 -- 172.21.15.26:0/1833400496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f958406a700 con 0x7f95840ce5c0 2024-04-26T21:08:44.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.940+0000 7fb6fde00640 1 -- 172.21.15.26:0/2508509653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb6f80cf220 con 0x7fb6f8160670 2024-04-26T21:08:44.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.939+0000 7f1439c00640 1 --2- 172.21.15.26:0/2612816864 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f141406c8f0 0x7f141406edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f14340b2670 tx=0x7f1428009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.940+0000 7f1431600640 1 -- 172.21.15.26:0/2612816864 <== osd.3 v2:172.21.15.26:6809/1280758061 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f142c01fec0 con 0x7f1414074a90 2024-04-26T21:08:44.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.941+0000 7f8e57200640 1 -- 172.21.15.26:0/2107785028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8e28001030 con 0x7f8e500d3dd0 2024-04-26T21:08:44.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.942+0000 7fede7e00640 1 -- 172.21.15.26:0/667080570 <== osd.4 v2:172.21.15.86:6816/1450967413 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fedd00775e0 con 0x7fedd0074b10 2024-04-26T21:08:44.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.944+0000 7fb6ef400640 1 -- 172.21.15.26:0/2508509653 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb6f001d070 con 0x7fb6f8160670 2024-04-26T21:08:44.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.946+0000 7f9581600640 1 -- 172.21.15.26:0/1833400496 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f957002a8e0 con 0x7f95840ce5c0 2024-04-26T21:08:44.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.946+0000 7fb6ef400640 1 --2- 172.21.15.26:0/2508509653 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb6d806c920 0x7fb6d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.945+0000 7f8e47e00640 1 -- 172.21.15.26:0/2107785028 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8e3c01b070 con 0x7f8e500d3dd0 2024-04-26T21:08:44.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.946+0000 7fb6ef400640 1 -- 172.21.15.26:0/2508509653 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb6f00583b0 con 0x7fb6f8160670 2024-04-26T21:08:44.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.946+0000 7fb6ef400640 1 --2- 172.21.15.26:0/2508509653 >> [v2:172.21.15.86:6808/2582257935,v1:172.21.15.86:6809/2582257935] conn(0x7fb6d80756c0 0x7fb6d8077b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.946+0000 7fb6ef400640 1 -- 172.21.15.26:0/2508509653 --> [v2:172.21.15.86:6808/2582257935,v1:172.21.15.86:6809/2582257935] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb6d8078230 con 0x7fb6d80756c0 2024-04-26T21:08:44.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.946+0000 7f8e47e00640 1 --2- 172.21.15.26:0/2107785028 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8e3006c9c0 0x7f8e3006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.947+0000 7fb6ef400640 1 -- 172.21.15.26:0/2508509653 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb6f0056d40 con 0x7fb6f8160670 2024-04-26T21:08:44.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.947+0000 7f8e47e00640 1 -- 172.21.15.26:0/2107785028 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8e3c08b530 con 0x7f8e500d3dd0 2024-04-26T21:08:44.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.947+0000 7f8e47e00640 1 --2- 172.21.15.26:0/2107785028 >> [v2:172.21.15.86:6800/1459341072,v1:172.21.15.86:6801/1459341072] conn(0x7f8e30074be0 0x7f8e30077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.947+0000 7f8e47e00640 1 -- 172.21.15.26:0/2107785028 --> [v2:172.21.15.86:6800/1459341072,v1:172.21.15.86:6801/1459341072] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8e3c014e50 con 0x7f8e30074be0 2024-04-26T21:08:44.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.947+0000 7f8e47e00640 1 -- 172.21.15.26:0/2107785028 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8e3c08b920 con 0x7f8e500d3dd0 2024-04-26T21:08:44.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.947+0000 7fb6f6000640 1 --2- 172.21.15.26:0/2508509653 >> [v2:172.21.15.86:6808/2582257935,v1:172.21.15.86:6809/2582257935] conn(0x7fb6d80756c0 0x7fb6d8077b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.947+0000 7f9581600640 1 --2- 172.21.15.26:0/1833400496 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f956406c9c0 0x7f956406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.947+0000 7f9581600640 1 -- 172.21.15.26:0/1833400496 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f95700acc00 con 0x7f95840ce5c0 2024-04-26T21:08:44.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.948+0000 7f9588a00640 1 --2- 172.21.15.26:0/1833400496 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f956406c9c0 0x7f956406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.948+0000 7f9581600640 1 --2- 172.21.15.26:0/1833400496 >> [v2:172.21.15.86:6823/4235789402,v1:172.21.15.86:6825/4235789402] conn(0x7f9564074be0 0x7f9564077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:44.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.948+0000 7f9581600640 1 -- 172.21.15.26:0/1833400496 --> [v2:172.21.15.86:6823/4235789402,v1:172.21.15.86:6825/4235789402] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9570001d30 con 0x7f9564074be0 2024-04-26T21:08:44.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.948+0000 7f8e55200640 1 --2- 172.21.15.26:0/2107785028 >> [v2:172.21.15.86:6800/1459341072,v1:172.21.15.86:6801/1459341072] conn(0x7f8e30074be0 0x7f8e30077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.948+0000 7f9581600640 1 -- 172.21.15.26:0/1833400496 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f95700acff0 con 0x7f95840ce5c0 2024-04-26T21:08:44.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.948+0000 7fb6f6000640 1 --2- 172.21.15.26:0/2508509653 >> [v2:172.21.15.86:6808/2582257935,v1:172.21.15.86:6809/2582257935] conn(0x7fb6d80756c0 0x7fb6d8077b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:44.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.948+0000 7f9589e00640 1 --2- 172.21.15.26:0/1833400496 >> [v2:172.21.15.86:6823/4235789402,v1:172.21.15.86:6825/4235789402] conn(0x7f9564074be0 0x7f9564077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:44.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.948+0000 7fb6f6a00640 1 --2- 172.21.15.26:0/2508509653 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb6d806c920 0x7fb6d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.949+0000 7f8e55200640 1 --2- 172.21.15.26:0/2107785028 >> [v2:172.21.15.86:6800/1459341072,v1:172.21.15.86:6801/1459341072] conn(0x7f8e30074be0 0x7f8e30077000 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.949+0000 7f9589e00640 1 --2- 172.21.15.26:0/1833400496 >> [v2:172.21.15.86:6823/4235789402,v1:172.21.15.86:6825/4235789402] conn(0x7f9564074be0 0x7f9564077000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.950+0000 7f8e4fe00640 1 --2- 172.21.15.26:0/2107785028 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8e3006c9c0 0x7f8e3006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.950+0000 7fb6f6a00640 1 --2- 172.21.15.26:0/2508509653 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb6d806c920 0x7fb6d806ede0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8005880 tx=0x7fb6e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.951+0000 7fb6ef400640 1 -- 172.21.15.26:0/2508509653 <== osd.7 v2:172.21.15.86:6808/2582257935 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb6d8078230 con 0x7fb6d80756c0 2024-04-26T21:08:45.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.951+0000 7f9588a00640 1 --2- 172.21.15.26:0/1833400496 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f956406c9c0 0x7f956406ee80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f95840ce1b0 tx=0x7f9578008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.951+0000 7f8e4fe00640 1 --2- 172.21.15.26:0/2107785028 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8e3006c9c0 0x7f8e3006ee80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8e40001840 tx=0x7f8e40009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.952+0000 7f8e47e00640 1 -- 172.21.15.26:0/2107785028 <== osd.5 v2:172.21.15.86:6800/1459341072 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e3c014e50 con 0x7f8e30074be0 2024-04-26T21:08:45.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.953+0000 7f9581600640 1 -- 172.21.15.26:0/1833400496 <== osd.6 v2:172.21.15.86:6823/4235789402 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9570001d30 con 0x7f9564074be0 2024-04-26T21:08:45.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.961+0000 7fed95000640 1 -- 172.21.15.26:0/314025201 --> [v2:172.21.15.26:6808/2924039897,v1:172.21.15.26:6811/2924039897] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fed68002dd0 con 0x7fed70074d10 2024-04-26T21:08:45.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.963+0000 7fed86000640 1 -- 172.21.15.26:0/314025201 <== osd.0 v2:172.21.15.26:6808/2924039897 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fed68002dd0 con 0x7fed70074d10 2024-04-26T21:08:45.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.964+0000 7fed95000640 1 -- 172.21.15.26:0/314025201 >> [v2:172.21.15.26:6808/2924039897,v1:172.21.15.26:6811/2924039897] conn(0x7fed70074d10 msgr2=0x7fed70077130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.964+0000 7fed95000640 1 --2- 172.21.15.26:0/314025201 >> [v2:172.21.15.26:6808/2924039897,v1:172.21.15.26:6811/2924039897] conn(0x7fed70074d10 0x7fed70077130 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.964+0000 7fed95000640 1 -- 172.21.15.26:0/314025201 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fed7006caf0 msgr2=0x7fed7006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.964+0000 7fed95000640 1 --2- 172.21.15.26:0/314025201 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fed7006caf0 0x7fed7006efb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fed800094a0 tx=0x7fed80009240 comp rx=0 tx=0).stop 2024-04-26T21:08:45.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.964+0000 7fed95000640 1 -- 172.21.15.26:0/314025201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed90163730 msgr2=0x7fed9014a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.964+0000 7fed95000640 1 --2- 172.21.15.26:0/314025201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed90163730 0x7fed9014a030 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fed900d1a10 tx=0x7fed7c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.964+0000 7fed8e800640 1 -- 172.21.15.26:0/314025201 reap_dead start 2024-04-26T21:08:45.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.966+0000 7fed95000640 1 -- 172.21.15.26:0/314025201 shutdown_connections 2024-04-26T21:08:45.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.966+0000 7fed95000640 1 -- 172.21.15.26:0/314025201 >> 172.21.15.26:0/314025201 conn(0x7fed900cd840 msgr2=0x7fed900b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.967+0000 7fb6fde00640 1 -- 172.21.15.26:0/2508509653 --> [v2:172.21.15.86:6808/2582257935,v1:172.21.15.86:6809/2582257935] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb6f80693d0 con 0x7fb6d80756c0 2024-04-26T21:08:45.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.968+0000 7fb6ef400640 1 -- 172.21.15.26:0/2508509653 <== osd.7 v2:172.21.15.86:6808/2582257935 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb6f80693d0 con 0x7fb6d80756c0 2024-04-26T21:08:45.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.969+0000 7fb6fde00640 1 -- 172.21.15.26:0/2508509653 >> [v2:172.21.15.86:6808/2582257935,v1:172.21.15.86:6809/2582257935] conn(0x7fb6d80756c0 msgr2=0x7fb6d8077b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.969+0000 7fb6fde00640 1 --2- 172.21.15.26:0/2508509653 >> [v2:172.21.15.86:6808/2582257935,v1:172.21.15.86:6809/2582257935] conn(0x7fb6d80756c0 0x7fb6d8077b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.970+0000 7fed95000640 1 -- 172.21.15.26:0/314025201 shutdown_connections 2024-04-26T21:08:45.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.970+0000 7fb6fde00640 1 -- 172.21.15.26:0/2508509653 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb6d806c920 msgr2=0x7fb6d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.970+0000 7fb6fde00640 1 --2- 172.21.15.26:0/2508509653 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb6d806c920 0x7fb6d806ede0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8005880 tx=0x7fb6e8009040 comp rx=0 tx=0).stop 2024-04-26T21:08:45.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.970+0000 7fb6fde00640 1 -- 172.21.15.26:0/2508509653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6f8160670 msgr2=0x7fb6f80b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.970+0000 7fb6fde00640 1 --2- 172.21.15.26:0/2508509653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6f8160670 0x7fb6f80b6e10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0007f20 tx=0x7fb6f000d560 comp rx=0 tx=0).stop 2024-04-26T21:08:45.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.970+0000 7fed95000640 1 -- 172.21.15.26:0/314025201 wait complete. 2024-04-26T21:08:45.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.970+0000 7fb6f7400640 1 -- 172.21.15.26:0/2508509653 reap_dead start 2024-04-26T21:08:45.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.970+0000 7fb6fde00640 1 -- 172.21.15.26:0/2508509653 shutdown_connections 2024-04-26T21:08:45.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.970+0000 7fb6fde00640 1 -- 172.21.15.26:0/2508509653 >> 172.21.15.26:0/2508509653 conn(0x7fb6f809e4b0 msgr2=0x7fb6f80d3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.970+0000 7fb6fde00640 1 -- 172.21.15.26:0/2508509653 shutdown_connections 2024-04-26T21:08:45.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.970+0000 7fb6fde00640 1 -- 172.21.15.26:0/2508509653 wait complete. 2024-04-26T21:08:45.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.973+0000 7f143c600640 1 -- 172.21.15.26:0/2612816864 --> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f14340caf50 con 0x7f1414074a90 2024-04-26T21:08:45.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.974+0000 7f1431600640 1 -- 172.21.15.26:0/2612816864 <== osd.3 v2:172.21.15.26:6809/1280758061 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f14340caf50 con 0x7f1414074a90 2024-04-26T21:08:45.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.974+0000 7f143c600640 1 -- 172.21.15.26:0/2612816864 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x7f1414074a90 msgr2=0x7f1414076eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.974+0000 7f143c600640 1 --2- 172.21.15.26:0/2612816864 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x7f1414074a90 0x7f1414076eb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.975+0000 7f143c600640 1 -- 172.21.15.26:0/2612816864 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f141406c8f0 msgr2=0x7f141406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.975+0000 7f143c600640 1 --2- 172.21.15.26:0/2612816864 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f141406c8f0 0x7f141406edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f14340b2670 tx=0x7f1428009190 comp rx=0 tx=0).stop 2024-04-26T21:08:45.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.975+0000 7f143c600640 1 -- 172.21.15.26:0/2612816864 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14341f8d70 msgr2=0x7f1434071320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.975+0000 7f143c600640 1 --2- 172.21.15.26:0/2612816864 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14341f8d70 0x7f1434071320 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f142c00c790 tx=0x7f142c00cc50 comp rx=0 tx=0).stop 2024-04-26T21:08:45.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.975+0000 7f143a600640 1 -- 172.21.15.26:0/2612816864 reap_dead start 2024-04-26T21:08:45.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.975+0000 7f28d6c00640 1 -- 172.21.15.26:0/1303997293 --> [v2:172.21.15.26:6824/2316564464,v1:172.21.15.26:6825/2316564464] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f28d006a700 con 0x7f28b0074cf0 2024-04-26T21:08:45.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.975+0000 7f143c600640 1 -- 172.21.15.26:0/2612816864 shutdown_connections 2024-04-26T21:08:45.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.975+0000 7f143c600640 1 -- 172.21.15.26:0/2612816864 >> 172.21.15.26:0/2612816864 conn(0x7f143409b9e0 msgr2=0x7f14340b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.975+0000 7f143c600640 1 -- 172.21.15.26:0/2612816864 shutdown_connections 2024-04-26T21:08:45.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.975+0000 7f143c600640 1 -- 172.21.15.26:0/2612816864 wait complete. 2024-04-26T21:08:45.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.976+0000 7fedf5800640 1 -- 172.21.15.26:0/667080570 --> [v2:172.21.15.86:6816/1450967413,v1:172.21.15.86:6817/1450967413] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fedf006eb70 con 0x7fedd0074b10 2024-04-26T21:08:45.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.976+0000 7f28c7e00640 1 -- 172.21.15.26:0/1303997293 <== osd.2 v2:172.21.15.26:6824/2316564464 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f28d006a700 con 0x7f28b0074cf0 2024-04-26T21:08:45.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.977+0000 7f28d6c00640 1 -- 172.21.15.26:0/1303997293 >> [v2:172.21.15.26:6824/2316564464,v1:172.21.15.26:6825/2316564464] conn(0x7f28b0074cf0 msgr2=0x7f28b0077110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.977+0000 7f28d6c00640 1 --2- 172.21.15.26:0/1303997293 >> [v2:172.21.15.26:6824/2316564464,v1:172.21.15.26:6825/2316564464] conn(0x7f28b0074cf0 0x7f28b0077110 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.977+0000 7fede7e00640 1 -- 172.21.15.26:0/667080570 <== osd.4 v2:172.21.15.86:6816/1450967413 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fedf006eb70 con 0x7fedd0074b10 2024-04-26T21:08:45.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.977+0000 7f28d6c00640 1 -- 172.21.15.26:0/1303997293 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f28b006caf0 msgr2=0x7f28b006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.977+0000 7f28d6c00640 1 --2- 172.21.15.26:0/1303997293 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f28b006caf0 0x7f28b006efb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f28c0034000 tx=0x7f28c002e000 comp rx=0 tx=0).stop 2024-04-26T21:08:45.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.977+0000 7f28d6c00640 1 -- 172.21.15.26:0/1303997293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28d00b4470 msgr2=0x7f28d00ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.977+0000 7f28d6c00640 1 --2- 172.21.15.26:0/1303997293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28d00b4470 0x7f28d00ce350 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f28c800efc0 tx=0x7f28c800c490 comp rx=0 tx=0).stop 2024-04-26T21:08:45.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.977+0000 7f28d4c00640 1 -- 172.21.15.26:0/1303997293 reap_dead start 2024-04-26T21:08:45.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.977+0000 7fedf5800640 1 -- 172.21.15.26:0/667080570 >> [v2:172.21.15.86:6816/1450967413,v1:172.21.15.86:6817/1450967413] conn(0x7fedd0074b10 msgr2=0x7fedd0076f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.977+0000 7fedf5800640 1 --2- 172.21.15.26:0/667080570 >> [v2:172.21.15.86:6816/1450967413,v1:172.21.15.86:6817/1450967413] conn(0x7fedd0074b10 0x7fedd0076f30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.978+0000 7fedf5800640 1 -- 172.21.15.26:0/667080570 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fedd006c8f0 msgr2=0x7fedd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.978+0000 7fedf5800640 1 --2- 172.21.15.26:0/667080570 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fedd006c8f0 0x7fedd006edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fede0045000 tx=0x7fede003f000 comp rx=0 tx=0).stop 2024-04-26T21:08:45.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.978+0000 7fedf5800640 1 -- 172.21.15.26:0/667080570 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fedf0073d30 msgr2=0x7fedf01f0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.978+0000 7fedf5800640 1 --2- 172.21.15.26:0/667080570 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fedf0073d30 0x7fedf01f0b20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7feddc00d800 tx=0x7feddc00dcc0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.978+0000 7fedf4e00640 1 -- 172.21.15.26:0/667080570 reap_dead start 2024-04-26T21:08:45.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.978+0000 7f28d6c00640 1 -- 172.21.15.26:0/1303997293 shutdown_connections 2024-04-26T21:08:45.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.978+0000 7f28d6c00640 1 -- 172.21.15.26:0/1303997293 >> 172.21.15.26:0/1303997293 conn(0x7f28d009e490 msgr2=0x7f28d006ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.978+0000 7fedf5800640 1 -- 172.21.15.26:0/667080570 shutdown_connections 2024-04-26T21:08:45.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.978+0000 7fedf5800640 1 -- 172.21.15.26:0/667080570 >> 172.21.15.26:0/667080570 conn(0x7fedf006c660 msgr2=0x7fedf00b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.978+0000 7fe72fe00640 1 -- 172.21.15.26:0/1605825689 --> [v2:172.21.15.26:6800/398686833,v1:172.21.15.26:6801/398686833] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe7281623a0 con 0x7fe708074b60 2024-04-26T21:08:45.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.981+0000 7fedf5800640 1 -- 172.21.15.26:0/667080570 shutdown_connections 2024-04-26T21:08:45.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.981+0000 7f28d6c00640 1 -- 172.21.15.26:0/1303997293 shutdown_connections 2024-04-26T21:08:45.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.981+0000 7f28d6c00640 1 -- 172.21.15.26:0/1303997293 wait complete. 2024-04-26T21:08:45.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.981+0000 7fedf5800640 1 -- 172.21.15.26:0/667080570 wait complete. 2024-04-26T21:08:45.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.984+0000 7fe725600640 1 -- 172.21.15.26:0/1605825689 <== osd.1 v2:172.21.15.26:6800/398686833 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe7281623a0 con 0x7fe708074b60 2024-04-26T21:08:45.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.985+0000 7fe72fe00640 1 -- 172.21.15.26:0/1605825689 >> [v2:172.21.15.26:6800/398686833,v1:172.21.15.26:6801/398686833] conn(0x7fe708074b60 msgr2=0x7fe708076f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.985+0000 7fe72fe00640 1 --2- 172.21.15.26:0/1605825689 >> [v2:172.21.15.26:6800/398686833,v1:172.21.15.26:6801/398686833] conn(0x7fe708074b60 0x7fe708076f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.986+0000 7fe72fe00640 1 -- 172.21.15.26:0/1605825689 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe70806c940 msgr2=0x7fe70806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.986+0000 7fe72fe00640 1 --2- 172.21.15.26:0/1605825689 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe70806c940 0x7fe70806ee00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe71c02a1c0 tx=0x7fe71c02e000 comp rx=0 tx=0).stop 2024-04-26T21:08:45.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.986+0000 7fe72fe00640 1 -- 172.21.15.26:0/1605825689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7280d3dd0 msgr2=0x7fe7281f9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.986+0000 7fe72fe00640 1 --2- 172.21.15.26:0/1605825689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7280d3dd0 0x7fe7281f9f40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe71400d5f0 tx=0x7fe71400dab0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.986+0000 7fe72de00640 1 -- 172.21.15.26:0/1605825689 reap_dead start 2024-04-26T21:08:45.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.987+0000 7fe72fe00640 1 -- 172.21.15.26:0/1605825689 shutdown_connections 2024-04-26T21:08:45.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.987+0000 7fe72fe00640 1 -- 172.21.15.26:0/1605825689 >> 172.21.15.26:0/1605825689 conn(0x7fe7280d51a0 msgr2=0x7fe7280b5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.987+0000 7fe72fe00640 1 -- 172.21.15.26:0/1605825689 shutdown_connections 2024-04-26T21:08:45.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.987+0000 7fe72fe00640 1 -- 172.21.15.26:0/1605825689 wait complete. 2024-04-26T21:08:45.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.988+0000 7f958be00640 1 -- 172.21.15.26:0/1833400496 --> [v2:172.21.15.86:6823/4235789402,v1:172.21.15.86:6825/4235789402] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f95840b9d30 con 0x7f9564074be0 2024-04-26T21:08:45.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.989+0000 7f9581600640 1 -- 172.21.15.26:0/1833400496 <== osd.6 v2:172.21.15.86:6823/4235789402 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f95840b9d30 con 0x7f9564074be0 2024-04-26T21:08:45.017 INFO:teuthology.orchestra.run.smithi026.stdout:55834574852 2024-04-26T21:08:45.018 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T21:08:45.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.994+0000 7f8e57200640 1 -- 172.21.15.26:0/2107785028 --> [v2:172.21.15.86:6800/1459341072,v1:172.21.15.86:6801/1459341072] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8e28002dd0 con 0x7f8e30074be0 2024-04-26T21:08:45.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.995+0000 7f8e47e00640 1 -- 172.21.15.26:0/2107785028 <== osd.5 v2:172.21.15.86:6800/1459341072 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8e28002dd0 con 0x7f8e30074be0 2024-04-26T21:08:45.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.996+0000 7f958be00640 1 -- 172.21.15.26:0/1833400496 >> [v2:172.21.15.86:6823/4235789402,v1:172.21.15.86:6825/4235789402] conn(0x7f9564074be0 msgr2=0x7f9564077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.996+0000 7f958be00640 1 --2- 172.21.15.26:0/1833400496 >> [v2:172.21.15.86:6823/4235789402,v1:172.21.15.86:6825/4235789402] conn(0x7f9564074be0 0x7f9564077000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.996+0000 7f958be00640 1 -- 172.21.15.26:0/1833400496 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f956406c9c0 msgr2=0x7f956406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.996+0000 7f958be00640 1 --2- 172.21.15.26:0/1833400496 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f956406c9c0 0x7f956406ee80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f95840ce1b0 tx=0x7f9578008040 comp rx=0 tx=0).stop 2024-04-26T21:08:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.996+0000 7f958be00640 1 -- 172.21.15.26:0/1833400496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95840ce5c0 msgr2=0x7f95840cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.996+0000 7f958be00640 1 --2- 172.21.15.26:0/1833400496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95840ce5c0 0x7f95840cc270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f957002a510 tx=0x7f9570004730 comp rx=0 tx=0).stop 2024-04-26T21:08:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.996+0000 7f9589e00640 1 -- 172.21.15.26:0/1833400496 reap_dead start 2024-04-26T21:08:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.996+0000 7f958be00640 1 -- 172.21.15.26:0/1833400496 shutdown_connections 2024-04-26T21:08:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.996+0000 7f958be00640 1 -- 172.21.15.26:0/1833400496 >> 172.21.15.26:0/1833400496 conn(0x7f95840d51a0 msgr2=0x7f95841666b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.996+0000 7f8e57200640 1 -- 172.21.15.26:0/2107785028 >> [v2:172.21.15.86:6800/1459341072,v1:172.21.15.86:6801/1459341072] conn(0x7f8e30074be0 msgr2=0x7f8e30077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.996+0000 7f8e57200640 1 --2- 172.21.15.26:0/2107785028 >> [v2:172.21.15.86:6800/1459341072,v1:172.21.15.86:6801/1459341072] conn(0x7f8e30074be0 0x7f8e30077000 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.021 INFO:teuthology.orchestra.run.smithi026.stdout:55834574854 2024-04-26T21:08:45.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.997+0000 7f8e57200640 1 -- 172.21.15.26:0/2107785028 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8e3006c9c0 msgr2=0x7f8e3006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.997+0000 7f8e57200640 1 --2- 172.21.15.26:0/2107785028 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8e3006c9c0 0x7f8e3006ee80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8e40001840 tx=0x7f8e40009040 comp rx=0 tx=0).stop 2024-04-26T21:08:45.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.997+0000 7f8e57200640 1 -- 172.21.15.26:0/2107785028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e500d3dd0 msgr2=0x7f8e50160b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.997+0000 7f8e57200640 1 --2- 172.21.15.26:0/2107785028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e500d3dd0 0x7f8e50160b70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00c750 tx=0x7f8e3c00cc10 comp rx=0 tx=0).stop 2024-04-26T21:08:45.022 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T21:08:45.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.997+0000 7f8e55200640 1 -- 172.21.15.26:0/2107785028 reap_dead start 2024-04-26T21:08:45.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.997+0000 7f958be00640 1 -- 172.21.15.26:0/1833400496 shutdown_connections 2024-04-26T21:08:45.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.997+0000 7f958be00640 1 -- 172.21.15.26:0/1833400496 wait complete. 2024-04-26T21:08:45.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.997+0000 7f8e57200640 1 -- 172.21.15.26:0/2107785028 shutdown_connections 2024-04-26T21:08:45.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.997+0000 7f8e57200640 1 -- 172.21.15.26:0/2107785028 >> 172.21.15.26:0/2107785028 conn(0x7f8e500d51a0 msgr2=0x7f8e500b5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.997+0000 7f8e57200640 1 -- 172.21.15.26:0/2107785028 shutdown_connections 2024-04-26T21:08:45.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:44.998+0000 7f8e57200640 1 -- 172.21.15.26:0/2107785028 wait complete. 2024-04-26T21:08:45.024 INFO:teuthology.orchestra.run.smithi026.stdout:55834574852 2024-04-26T21:08:45.024 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T21:08:45.025 INFO:teuthology.orchestra.run.smithi026.stdout:55834574852 2024-04-26T21:08:45.025 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T21:08:45.026 INFO:teuthology.orchestra.run.smithi026.stdout:55834574852 2024-04-26T21:08:45.026 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T21:08:45.027 INFO:teuthology.orchestra.run.smithi026.stdout:55834574853 2024-04-26T21:08:45.027 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T21:08:45.028 INFO:teuthology.orchestra.run.smithi026.stdout:55834574852 2024-04-26T21:08:45.028 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T21:08:45.029 INFO:teuthology.orchestra.run.smithi026.stdout:55834574852 2024-04-26T21:08:45.029 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T21:08:45.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.347+0000 7f145b400640 1 Processor -- start 2024-04-26T21:08:45.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.348+0000 7f145b400640 1 -- start start 2024-04-26T21:08:45.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.350+0000 7f145b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14540b6c50 0x7f145406e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.350+0000 7f145b400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1454072d60 con 0x7f145406e280 2024-04-26T21:08:45.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.350+0000 7f145b400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1454073000 con 0x7f14540cf960 2024-04-26T21:08:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.350+0000 7f145b400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f14540732e0 con 0x7f14540b6c50 2024-04-26T21:08:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.350+0000 7f1458a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f145406e280 0x7f14540ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44790/0 (socket says 172.21.15.26:44790) 2024-04-26T21:08:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.350+0000 7f1453e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14540b6c50 0x7f145406e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.350+0000 7f1458a00640 1 -- 172.21.15.26:0/1840896296 learned_addr learned my addr 172.21.15.26:0/1840896296 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:45.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.351+0000 7f1453400640 1 -- 172.21.15.26:0/1840896296 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4066647701 0 0) 0x7f1454072d60 con 0x7f145406e280 2024-04-26T21:08:45.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.351+0000 7f1453400640 1 -- 172.21.15.26:0/1840896296 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1438003520 con 0x7f145406e280 2024-04-26T21:08:45.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.352+0000 7f1453e00640 1 -- 172.21.15.26:0/1840896296 >> v1:172.21.15.86:6789/0 conn(0x7f14540cf960 legacy=0x7f1454164e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:45.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.352+0000 7f1453e00640 1 -- 172.21.15.26:0/1840896296 >> v1:172.21.15.26:6789/0 conn(0x7f145406e280 legacy=0x7f14540ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.352+0000 7f1453e00640 1 -- 172.21.15.26:0/1840896296 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14540735c0 con 0x7f14540b6c50 2024-04-26T21:08:45.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.353+0000 7f1453e00640 1 --2- 172.21.15.26:0/1840896296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14540b6c50 0x7f145406e660 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f14440097a0 tx=0x7f144402a140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=451a39cd6529e9bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.354+0000 7f1453400640 1 -- 172.21.15.26:0/1840896296 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444037070 con 0x7f14540b6c50 2024-04-26T21:08:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.354+0000 7f1453400640 1 -- 172.21.15.26:0/1840896296 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f144402ae50 con 0x7f14540b6c50 2024-04-26T21:08:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.354+0000 7f1453400640 1 -- 172.21.15.26:0/1840896296 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444032660 con 0x7f14540b6c50 2024-04-26T21:08:45.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.356+0000 7f145b400640 1 -- 172.21.15.26:0/1840896296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14540b6c50 msgr2=0x7f145406e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.356+0000 7f145b400640 1 --2- 172.21.15.26:0/1840896296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14540b6c50 0x7f145406e660 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f14440097a0 tx=0x7f144402a140 comp rx=0 tx=0).stop 2024-04-26T21:08:45.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.356+0000 7f145b400640 1 -- 172.21.15.26:0/1840896296 shutdown_connections 2024-04-26T21:08:45.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.356+0000 7f145b400640 1 --2- 172.21.15.26:0/1840896296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14540b6c50 0x7f145406e660 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.356+0000 7f145b400640 1 -- 172.21.15.26:0/1840896296 >> 172.21.15.26:0/1840896296 conn(0x7f14540b2af0 msgr2=0x7f14540b2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.356+0000 7fc931200640 1 Processor -- start 2024-04-26T21:08:45.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.357+0000 7f145b400640 1 -- 172.21.15.26:0/1840896296 shutdown_connections 2024-04-26T21:08:45.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.357+0000 7f145b400640 1 -- 172.21.15.26:0/1840896296 wait complete. 2024-04-26T21:08:45.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.357+0000 7fc931200640 1 -- start start 2024-04-26T21:08:45.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.359+0000 7fc931200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc92c163730 0x7fc92c0d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.360+0000 7fc931200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc92c0d13e0 0x7fc92c06e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.359+0000 7f145b400640 1 Processor -- start 2024-04-26T21:08:45.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.360+0000 7f145b400640 1 -- start start 2024-04-26T21:08:45.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.360+0000 7fc931200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc92c06e280 con 0x7fc92c0b3df0 2024-04-26T21:08:45.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.360+0000 7fc931200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc92c167680 con 0x7fc92c163730 2024-04-26T21:08:45.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.360+0000 7fc931200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc92c167960 con 0x7fc92c0d13e0 2024-04-26T21:08:45.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.360+0000 7fc929400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc92c0b3df0 0x7fc92c0d0cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.26:34024/0 (socket says 172.21.15.26:34024) 2024-04-26T21:08:45.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.360+0000 7fc929400640 1 -- 172.21.15.26:0/1625719913 learned_addr learned my addr 172.21.15.26:0/1625719913 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:45.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.360+0000 7fc92a800640 1 --2- 172.21.15.26:0/1625719913 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc92c0d13e0 0x7fc92c06e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.361+0000 7fc929e00640 1 --2- 172.21.15.26:0/1625719913 >> v2:172.21.15.86:3300/0 conn(0x7fc92c163730 0x7fc92c0d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.361+0000 7f145b400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f145406e280 0x7f1454147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.361+0000 7fc92a800640 1 -- 172.21.15.26:0/1625719913 >> v2:172.21.15.86:3300/0 conn(0x7fc92c163730 msgr2=0x7fc92c0d0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:45.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.361+0000 7fc92a800640 1 --2- 172.21.15.26:0/1625719913 >> v2:172.21.15.86:3300/0 conn(0x7fc92c163730 0x7fc92c0d0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.361+0000 7fc92a800640 1 -- 172.21.15.26:0/1625719913 >> v1:172.21.15.86:6789/0 conn(0x7fc92c0b3df0 legacy=0x7fc92c0d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.361+0000 7fc92a800640 1 -- 172.21.15.26:0/1625719913 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92c167c40 con 0x7fc92c0d13e0 2024-04-26T21:08:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.361+0000 7f145b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14540b6c50 0x7f1454155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.361+0000 7f145b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14540cf960 0x7f14541e44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.361+0000 7f145b400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f14540b3790 con 0x7f14540cf960 2024-04-26T21:08:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.361+0000 7f145b400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14540b3610 con 0x7f14540b6c50 2024-04-26T21:08:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.361+0000 7f145b400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f145406eba0 con 0x7f145406e280 2024-04-26T21:08:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.362+0000 7f1459400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14540cf960 0x7f14541e44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.362+0000 7f1459400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14540cf960 0x7f14541e44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34634/0 (socket says 172.21.15.26:34634) 2024-04-26T21:08:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.362+0000 7f1459400640 1 -- 172.21.15.26:0/715308174 learned_addr learned my addr 172.21.15.26:0/715308174 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.362+0000 7f1458a00640 1 --2- 172.21.15.26:0/715308174 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f145406e280 0x7f1454147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.362+0000 7f1453e00640 1 --2- 172.21.15.26:0/715308174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14540b6c50 0x7f1454155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.363+0000 7f1459400640 1 -- 172.21.15.26:0/715308174 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f145406e280 msgr2=0x7f1454147280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.363+0000 7f1459400640 1 --2- 172.21.15.26:0/715308174 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f145406e280 0x7f1454147280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.363+0000 7fe956600640 1 Processor -- start 2024-04-26T21:08:45.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.363+0000 7f1459400640 1 -- 172.21.15.26:0/715308174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14540b6c50 msgr2=0x7f1454155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.363+0000 7f1459400640 1 --2- 172.21.15.26:0/715308174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14540b6c50 0x7f1454155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.363+0000 7f1459400640 1 -- 172.21.15.26:0/715308174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1454156290 con 0x7f14540cf960 2024-04-26T21:08:45.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.363+0000 7f1459400640 1 --2- 172.21.15.26:0/715308174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14540cf960 0x7f14541e44b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f144000b440 tx=0x7f144000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.364+0000 7f1450c00640 1 -- 172.21.15.26:0/715308174 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440011830 con 0x7f14540cf960 2024-04-26T21:08:45.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.364+0000 7f1450c00640 1 -- 172.21.15.26:0/715308174 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1440016070 con 0x7f14540cf960 2024-04-26T21:08:45.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.364+0000 7f1450c00640 1 -- 172.21.15.26:0/715308174 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440011bf0 con 0x7f14540cf960 2024-04-26T21:08:45.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.364+0000 7f145b400640 1 -- 172.21.15.26:0/715308174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1454156570 con 0x7f14540cf960 2024-04-26T21:08:45.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.364+0000 7f145b400640 1 -- 172.21.15.26:0/715308174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1454156b30 con 0x7f14540cf960 2024-04-26T21:08:45.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.363+0000 7fe956600640 1 -- start start 2024-04-26T21:08:45.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.365+0000 7fe956600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe950163730 0x7fe9500d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.365+0000 7fe956600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe9500b3df0 0x7fe9500d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.365+0000 7fe956600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe950167180 0x7fe9500d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.365+0000 7fe956600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fe95006e280 con 0x7fe950167180 2024-04-26T21:08:45.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.365+0000 7fe956600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe950167680 con 0x7fe9500b3df0 2024-04-26T21:08:45.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.365+0000 7fe956600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe950167960 con 0x7fe950163730 2024-04-26T21:08:45.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.366+0000 7fe94f400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe950163730 0x7fe9500d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.366+0000 7fd2a6600640 1 Processor -- start 2024-04-26T21:08:45.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.366+0000 7fe94f400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe950163730 0x7fe9500d0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.26:46020/0 (socket says 172.21.15.26:46020) 2024-04-26T21:08:45.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.366+0000 7fe94f400640 1 -- 172.21.15.26:0/3047801865 learned_addr learned my addr 172.21.15.26:0/3047801865 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:45.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.367+0000 7fe94fe00640 1 --2- 172.21.15.26:0/3047801865 >> v2:172.21.15.26:3300/0 conn(0x7fe950167180 0x7fe9500d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.367+0000 7fe94ea00640 1 --2- 172.21.15.26:0/3047801865 >> v2:172.21.15.86:3300/0 conn(0x7fe9500b3df0 0x7fe9500d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.367+0000 7fe94f400640 1 -- 172.21.15.26:0/3047801865 >> v2:172.21.15.86:3300/0 conn(0x7fe9500b3df0 msgr2=0x7fe9500d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:45.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.367+0000 7fe94f400640 1 --2- 172.21.15.26:0/3047801865 >> v2:172.21.15.86:3300/0 conn(0x7fe9500b3df0 0x7fe9500d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.367+0000 7fe94f400640 1 -- 172.21.15.26:0/3047801865 >> v2:172.21.15.26:3300/0 conn(0x7fe950167180 msgr2=0x7fe9500d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:45.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.367+0000 7fe94f400640 1 --2- 172.21.15.26:0/3047801865 >> v2:172.21.15.26:3300/0 conn(0x7fe950167180 0x7fe9500d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.367+0000 7fe94f400640 1 -- 172.21.15.26:0/3047801865 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe950167c40 con 0x7fe950163730 2024-04-26T21:08:45.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.367+0000 7fe94ea00640 1 --2- 172.21.15.26:0/3047801865 >> v2:172.21.15.86:3300/0 conn(0x7fe9500b3df0 0x7fe9500d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:45.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.366+0000 7f145b400640 1 -- 172.21.15.26:0/715308174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f145406a700 con 0x7f14540cf960 2024-04-26T21:08:45.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.368+0000 7fe94fe00640 1 --2- 172.21.15.26:0/3047801865 >> v2:172.21.15.26:3300/0 conn(0x7fe950167180 0x7fe9500d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:45.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.366+0000 7fd2a6600640 1 -- start start 2024-04-26T21:08:45.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.368+0000 7fd2a6600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2a01522f0 0x7fd2a006fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.368+0000 7fd2a6600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2a0070510 con 0x7fd2a00b5ea0 2024-04-26T21:08:45.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.368+0000 7f678f600640 1 Processor -- start 2024-04-26T21:08:45.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.369+0000 7fd2a6600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2a015b9a0 con 0x7fd2a0073790 2024-04-26T21:08:45.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.369+0000 7fd29f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2a01522f0 0x7fd2a006fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.369+0000 7fe94f400640 1 --2- 172.21.15.26:0/3047801865 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe950163730 0x7fe9500d0410 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe9500ce890 tx=0x7fe94802a690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6b652b29d0807b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.369+0000 7fd29f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2a01522f0 0x7fd2a006fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34662/0 (socket says 172.21.15.26:34662) 2024-04-26T21:08:45.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.369+0000 7fd29f400640 1 -- 172.21.15.26:0/139517585 learned_addr learned my addr 172.21.15.26:0/139517585 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:45.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.369+0000 7fd2a6600640 1 -- 172.21.15.26:0/139517585 --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fd2a015bff0 con 0x7fd2a01522f0 2024-04-26T21:08:45.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.369+0000 7fe94e000640 1 -- 172.21.15.26:0/3047801865 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe948036070 con 0x7fe950163730 2024-04-26T21:08:45.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.370+0000 7fe94e000640 1 -- 172.21.15.26:0/3047801865 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe948031600 con 0x7fe950163730 2024-04-26T21:08:45.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.370+0000 7fe94e000640 1 -- 172.21.15.26:0/3047801865 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9480319d0 con 0x7fe950163730 2024-04-26T21:08:45.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.370+0000 7fd29e000640 1 -- 172.21.15.26:0/139517585 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4223812274 0 0) 0x7fd2a015b9a0 con 0x7fd2a0073790 2024-04-26T21:08:45.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.370+0000 7fd29f400640 1 -- 172.21.15.26:0/139517585 >> v1:172.21.15.86:6789/0 conn(0x7fd2a0073790 legacy=0x7fd2a00b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.370+0000 7fd29f400640 1 -- 172.21.15.26:0/139517585 >> v1:172.21.15.26:6789/0 conn(0x7fd2a00b5ea0 legacy=0x7fd2a0073080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.370+0000 7fd29f400640 1 -- 172.21.15.26:0/139517585 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a015bc80 con 0x7fd2a01522f0 2024-04-26T21:08:45.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.367+0000 7fc92a800640 1 --2- 172.21.15.26:0/1625719913 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc92c0d13e0 0x7fc92c06e760 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc9180097a0 tx=0x7fc91802a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=255d84c227f84fb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.369+0000 7f678f600640 1 -- start start 2024-04-26T21:08:45.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.370+0000 7f678f600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f678806e280 0x7f67880ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.371+0000 7f678f600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f67880b6c50 0x7f678806e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.371+0000 7f678f600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f678806eba0 0x7f67880ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.371+0000 7f678f600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f6788072d60 con 0x7f678806e280 2024-04-26T21:08:45.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.371+0000 7f678f600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6788073000 con 0x7f67880b6c50 2024-04-26T21:08:45.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.371+0000 7f678f600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f67880732e0 con 0x7f678806eba0 2024-04-26T21:08:45.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.371+0000 7fe956600640 1 -- 172.21.15.26:0/3047801865 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe950163730 msgr2=0x7fe9500d0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.371+0000 7fe956600640 1 --2- 172.21.15.26:0/3047801865 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe950163730 0x7fe9500d0410 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe9500ce890 tx=0x7fe94802a690 comp rx=0 tx=0).stop 2024-04-26T21:08:45.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.371+0000 7f678cc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f678806e280 0x7f67880ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.371+0000 7fc928a00640 1 -- 172.21.15.26:0/1625719913 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918037070 con 0x7fc92c0d13e0 2024-04-26T21:08:45.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.372+0000 7fe956600640 1 -- 172.21.15.26:0/3047801865 shutdown_connections 2024-04-26T21:08:45.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.372+0000 7fe956600640 1 --2- 172.21.15.26:0/3047801865 >> v2:172.21.15.26:3300/0 conn(0x7fe950167180 0x7fe9500d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.372+0000 7fe956600640 1 --2- 172.21.15.26:0/3047801865 >> v2:172.21.15.86:3300/0 conn(0x7fe9500b3df0 0x7fe9500d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.372+0000 7fe956600640 1 --2- 172.21.15.26:0/3047801865 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe950163730 0x7fe9500d0410 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.372+0000 7fe956600640 1 -- 172.21.15.26:0/3047801865 >> 172.21.15.26:0/3047801865 conn(0x7fe9500cd840 msgr2=0x7fe9500cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.372+0000 7fd29f400640 1 --2- 172.21.15.26:0/139517585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a01522f0 0x7fd2a006fbc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd28800b1e0 tx=0x7fd28802a9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3aed7cfa930023 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.372+0000 7f678cc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f678806e280 0x7f67880ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34672/0 (socket says 172.21.15.26:34672) 2024-04-26T21:08:45.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.372+0000 7f678cc00640 1 -- 172.21.15.26:0/3323075106 learned_addr learned my addr 172.21.15.26:0/3323075106 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:45.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.372+0000 7f6787e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f67880b6c50 0x7f678806e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.372+0000 7fe956600640 1 -- 172.21.15.26:0/3047801865 shutdown_connections 2024-04-26T21:08:45.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.372+0000 7fe956600640 1 -- 172.21.15.26:0/3047801865 wait complete. 2024-04-26T21:08:45.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.372+0000 7f678d600640 1 --2- 172.21.15.26:0/3323075106 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f678806eba0 0x7f67880ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.372+0000 7f1450c00640 1 -- 172.21.15.26:0/715308174 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f144001b460 con 0x7f14540cf960 2024-04-26T21:08:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.372+0000 7fc928a00640 1 -- 172.21.15.26:0/1625719913 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc91802ad10 con 0x7fc92c0d13e0 2024-04-26T21:08:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.372+0000 7fc928a00640 1 -- 172.21.15.26:0/1625719913 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918032660 con 0x7fc92c0d13e0 2024-04-26T21:08:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.373+0000 7fc931200640 1 -- 172.21.15.26:0/1625719913 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc92c0d13e0 msgr2=0x7fc92c06e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.373+0000 7fc931200640 1 --2- 172.21.15.26:0/1625719913 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc92c0d13e0 0x7fc92c06e760 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc9180097a0 tx=0x7fc91802a080 comp rx=0 tx=0).stop 2024-04-26T21:08:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.374+0000 7fc931200640 1 -- 172.21.15.26:0/1625719913 shutdown_connections 2024-04-26T21:08:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.374+0000 7fc931200640 1 --2- 172.21.15.26:0/1625719913 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc92c0d13e0 0x7fc92c06e760 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.374+0000 7fc931200640 1 --2- 172.21.15.26:0/1625719913 >> v2:172.21.15.86:3300/0 conn(0x7fc92c163730 0x7fc92c0d0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.374+0000 7fc931200640 1 -- 172.21.15.26:0/1625719913 >> 172.21.15.26:0/1625719913 conn(0x7fc92c0cd840 msgr2=0x7fc92c0cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.373+0000 7f6787e00640 1 -- 172.21.15.26:0/3323075106 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f678806eba0 msgr2=0x7f67880ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.373+0000 7f6787e00640 1 --2- 172.21.15.26:0/3323075106 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f678806eba0 0x7f67880ce950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.373+0000 7f6787e00640 1 -- 172.21.15.26:0/3323075106 >> v2:172.21.15.26:3300/0 conn(0x7f678806e280 msgr2=0x7f67880ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:45.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.374+0000 7f1450c00640 1 --2- 172.21.15.26:0/715308174 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f143406c9c0 0x7f143406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.374+0000 7f1450c00640 1 -- 172.21.15.26:0/715308174 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f144008b810 con 0x7f14540cf960 2024-04-26T21:08:45.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.374+0000 7f1458a00640 1 --2- 172.21.15.26:0/715308174 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f143406c9c0 0x7f143406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.373+0000 7f6787e00640 1 --2- 172.21.15.26:0/3323075106 >> v2:172.21.15.26:3300/0 conn(0x7f678806e280 0x7f67880ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.373+0000 7f6787e00640 1 -- 172.21.15.26:0/3323075106 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67880735c0 con 0x7f67880b6c50 2024-04-26T21:08:45.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.374+0000 7fc931200640 1 -- 172.21.15.26:0/1625719913 shutdown_connections 2024-04-26T21:08:45.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.374+0000 7fc931200640 1 -- 172.21.15.26:0/1625719913 wait complete. 2024-04-26T21:08:45.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.374+0000 7f7072000640 1 Processor -- start 2024-04-26T21:08:45.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.375+0000 7f6787e00640 1 --2- 172.21.15.26:0/3323075106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67880b6c50 0x7f678806e660 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f67880b3a80 tx=0x7f677402e550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=980df07996181722 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.375+0000 7f7072000640 1 -- start start 2024-04-26T21:08:45.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.376+0000 7f1458a00640 1 --2- 172.21.15.26:0/715308174 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f143406c9c0 0x7f143406ee80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f144c005c00 tx=0x7f144c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.376+0000 7fe956600640 1 Processor -- start 2024-04-26T21:08:45.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.376+0000 7f6787400640 1 -- 172.21.15.26:0/3323075106 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677403a070 con 0x7f67880b6c50 2024-04-26T21:08:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.376+0000 7f6787400640 1 -- 172.21.15.26:0/3323075106 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6774033710 con 0x7f67880b6c50 2024-04-26T21:08:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.376+0000 7f6787400640 1 -- 172.21.15.26:0/3323075106 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6774033ac0 con 0x7f67880b6c50 2024-04-26T21:08:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.376+0000 7fc931200640 1 Processor -- start 2024-04-26T21:08:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.377+0000 7f678f600640 1 -- 172.21.15.26:0/3323075106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67880b6c50 msgr2=0x7f678806e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.377+0000 7f678f600640 1 --2- 172.21.15.26:0/3323075106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67880b6c50 0x7f678806e660 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f67880b3a80 tx=0x7f677402e550 comp rx=0 tx=0).stop 2024-04-26T21:08:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.377+0000 7f7072000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f706c0d1fd0 0x7f706c16a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.377+0000 7f7072000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f706c0b4850 con 0x7f706c0cfd30 2024-04-26T21:08:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.377+0000 7f7072000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f706c0d02f0 con 0x7f706c0b4470 2024-04-26T21:08:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.377+0000 7f7072000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f706c162fd0 con 0x7f706c0d1fd0 2024-04-26T21:08:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.377+0000 7f678f600640 1 -- 172.21.15.26:0/3323075106 shutdown_connections 2024-04-26T21:08:45.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.377+0000 7fc931200640 1 -- start start 2024-04-26T21:08:45.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.377+0000 7f678f600640 1 --2- 172.21.15.26:0/3323075106 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f678806eba0 0x7f67880ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.377+0000 7f678f600640 1 --2- 172.21.15.26:0/3323075106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67880b6c50 0x7f678806e660 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.377+0000 7f678f600640 1 --2- 172.21.15.26:0/3323075106 >> v2:172.21.15.26:3300/0 conn(0x7f678806e280 0x7f67880ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.377+0000 7f678f600640 1 -- 172.21.15.26:0/3323075106 >> 172.21.15.26:0/3323075106 conn(0x7f67880b2af0 msgr2=0x7f67880b2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.376+0000 7fe956600640 1 -- start start 2024-04-26T21:08:45.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.377+0000 7f678f600640 1 -- 172.21.15.26:0/3323075106 shutdown_connections 2024-04-26T21:08:45.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.377+0000 7fe956600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9500b3df0 0x7fe95014d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.377+0000 7fe956600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe950163730 0x7fe950163b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.377+0000 7fe956600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe950167180 0x7fe9500722c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.377+0000 7fe956600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe9500d0ad0 con 0x7fe9500b3df0 2024-04-26T21:08:45.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.377+0000 7fe956600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe9500d0950 con 0x7fe950163730 2024-04-26T21:08:45.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.377+0000 7fe956600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe9500ce950 con 0x7fe950167180 2024-04-26T21:08:45.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.378+0000 7f678f600640 1 -- 172.21.15.26:0/3323075106 wait complete. 2024-04-26T21:08:45.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.378+0000 7fc931200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc92c0b3df0 0x7fc92c1f9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.378+0000 7fe94f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9500b3df0 0x7fe95014d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.378+0000 7fc931200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc92c0d13e0 0x7fc92c0b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.378+0000 7fc931200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc92c163730 0x7fc92c0b2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.378+0000 7fc931200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc92c0d0ad0 con 0x7fc92c163730 2024-04-26T21:08:45.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.378+0000 7fc931200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc92c0d0950 con 0x7fc92c0b3df0 2024-04-26T21:08:45.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.378+0000 7fc931200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc92c167180 con 0x7fc92c0d13e0 2024-04-26T21:08:45.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.378+0000 7fe94ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe950163730 0x7fe950163b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.378+0000 7fe94ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe950163730 0x7fe950163b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.26:41850/0 (socket says 172.21.15.26:41850) 2024-04-26T21:08:45.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.378+0000 7fe94ea00640 1 -- 172.21.15.26:0/215313176 learned_addr learned my addr 172.21.15.26:0/215313176 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:45.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.378+0000 7f706be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f706c0d1fd0 0x7f706c16a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.378+0000 7f706be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f706c0d1fd0 0x7f706c16a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.26:41840/0 (socket says 172.21.15.26:41840) 2024-04-26T21:08:45.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.378+0000 7f706be00640 1 -- 172.21.15.26:0/1718190725 learned_addr learned my addr 172.21.15.26:0/1718190725 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:45.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.378+0000 7f706b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f706c0b4470 0x7f706c0d3ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.26:34038/0 (socket says 172.21.15.26:34038) 2024-04-26T21:08:45.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.379+0000 7fc929e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc92c0b3df0 0x7fc92c1f9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.379+0000 7fc929400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc92c0d13e0 0x7fc92c0b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.379+0000 7fc92a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc92c163730 0x7fc92c0b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.379+0000 7fc929e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc92c0b3df0 0x7fc92c1f9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.26:41864/0 (socket says 172.21.15.26:41864) 2024-04-26T21:08:45.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.379+0000 7fc929e00640 1 -- 172.21.15.26:0/1695282898 learned_addr learned my addr 172.21.15.26:0/1695282898 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:45.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.379+0000 7fe94fe00640 1 --2- 172.21.15.26:0/215313176 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe950167180 0x7fe9500722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.379+0000 7f706be00640 1 -- 172.21.15.26:0/1718190725 >> v1:172.21.15.86:6789/0 conn(0x7f706c0b4470 legacy=0x7f706c0d3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:45.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.379+0000 7fe94ea00640 1 -- 172.21.15.26:0/215313176 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe950167180 msgr2=0x7fe9500722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.379+0000 7fe94ea00640 1 --2- 172.21.15.26:0/215313176 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe950167180 0x7fe9500722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.379+0000 7fe94ea00640 1 -- 172.21.15.26:0/215313176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9500b3df0 msgr2=0x7fe95014d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.380+0000 7fc92a800640 1 -- 172.21.15.26:0/1695282898 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc92c0d13e0 msgr2=0x7fc92c0b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.380+0000 7fc92a800640 1 --2- 172.21.15.26:0/1695282898 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc92c0d13e0 0x7fc92c0b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.380+0000 7fc92a800640 1 -- 172.21.15.26:0/1695282898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc92c0b3df0 msgr2=0x7fc92c1f9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.380+0000 7fc92a800640 1 --2- 172.21.15.26:0/1695282898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc92c0b3df0 0x7fc92c1f9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.380+0000 7fc92a800640 1 -- 172.21.15.26:0/1695282898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92c162b90 con 0x7fc92c163730 2024-04-26T21:08:45.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.379+0000 7fe94ea00640 1 --2- 172.21.15.26:0/215313176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9500b3df0 0x7fe95014d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.380+0000 7f706be00640 1 -- 172.21.15.26:0/1718190725 >> v1:172.21.15.26:6789/0 conn(0x7f706c0cfd30 legacy=0x7f706c0d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.379+0000 7fe94ea00640 1 -- 172.21.15.26:0/215313176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95014cc20 con 0x7fe950163730 2024-04-26T21:08:45.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.380+0000 7f706be00640 1 -- 172.21.15.26:0/1718190725 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f706c0d05d0 con 0x7f706c0d1fd0 2024-04-26T21:08:45.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.380+0000 7f706a000640 1 -- 172.21.15.26:0/1718190725 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3624437679 0 0) 0x7f706c0b4850 con 0x7f706c0cfd30 2024-04-26T21:08:45.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.381+0000 7f678f600640 1 Processor -- start 2024-04-26T21:08:45.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.381+0000 7f1450c00640 1 -- 172.21.15.26:0/715308174 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1440059c20 con 0x7f14540cf960 2024-04-26T21:08:45.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.381+0000 7f678f600640 1 -- start start 2024-04-26T21:08:45.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.381+0000 7fc92a800640 1 --2- 172.21.15.26:0/1695282898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc92c163730 0x7fc92c0b2010 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc92c06e450 tx=0x7fc9180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.381+0000 7fd29e000640 1 -- 172.21.15.26:0/139517585 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288033020 con 0x7fd2a01522f0 2024-04-26T21:08:45.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.381+0000 7fd29e000640 1 -- 172.21.15.26:0/139517585 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd288007ce0 con 0x7fd2a01522f0 2024-04-26T21:08:45.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.381+0000 7fd29e000640 1 -- 172.21.15.26:0/139517585 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2880092f0 con 0x7fd2a01522f0 2024-04-26T21:08:45.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.381+0000 7fe94ea00640 1 --2- 172.21.15.26:0/215313176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe950163730 0x7fe950163b10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00c7f0 tx=0x7fe93c00ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.382+0000 7fc922000640 1 -- 172.21.15.26:0/1695282898 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918037070 con 0x7fc92c163730 2024-04-26T21:08:45.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.382+0000 7fc922000640 1 -- 172.21.15.26:0/1695282898 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc918038040 con 0x7fc92c163730 2024-04-26T21:08:45.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.382+0000 7fc922000640 1 -- 172.21.15.26:0/1695282898 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9180044b0 con 0x7fc92c163730 2024-04-26T21:08:45.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.382+0000 7fe947400640 1 -- 172.21.15.26:0/215313176 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c015070 con 0x7fe950163730 2024-04-26T21:08:45.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.382+0000 7fc931200640 1 -- 172.21.15.26:0/1695282898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc92c1fa120 con 0x7fc92c163730 2024-04-26T21:08:45.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.382+0000 7fd2a6600640 1 -- 172.21.15.26:0/139517585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a01522f0 msgr2=0x7fd2a006fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.382+0000 7fd2a6600640 1 --2- 172.21.15.26:0/139517585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a01522f0 0x7fd2a006fbc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd28800b1e0 tx=0x7fd28802a9a0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.382+0000 7f88e3400640 1 Processor -- start 2024-04-26T21:08:45.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.382+0000 7fe947400640 1 -- 172.21.15.26:0/215313176 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe93c007a30 con 0x7fe950163730 2024-04-26T21:08:45.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.382+0000 7fe947400640 1 -- 172.21.15.26:0/215313176 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c015070 con 0x7fe950163730 2024-04-26T21:08:45.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.382+0000 7fd2a6600640 1 -- 172.21.15.26:0/139517585 shutdown_connections 2024-04-26T21:08:45.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.382+0000 7fd2a6600640 1 --2- 172.21.15.26:0/139517585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a01522f0 0x7fd2a006fbc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.382+0000 7fd2a6600640 1 -- 172.21.15.26:0/139517585 >> 172.21.15.26:0/139517585 conn(0x7fd2a006ca30 msgr2=0x7fd2a00b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.383+0000 7fd2a6600640 1 -- 172.21.15.26:0/139517585 shutdown_connections 2024-04-26T21:08:45.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.383+0000 7fd2a6600640 1 -- 172.21.15.26:0/139517585 wait complete. 2024-04-26T21:08:45.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.383+0000 7f678f600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f678806e280 0x7f678814ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.383+0000 7f678f600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678806eba0 0x7f678814f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.383+0000 7f678f600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67880b6c50 0x7f67881f0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.383+0000 7f88e3400640 1 -- start start 2024-04-26T21:08:45.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.382+0000 7fc931200640 1 -- 172.21.15.26:0/1695282898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc92c0b4970 con 0x7fc92c163730 2024-04-26T21:08:45.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.384+0000 7fe956600640 1 -- 172.21.15.26:0/215313176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe950164410 con 0x7fe950163730 2024-04-26T21:08:45.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.384+0000 7fe956600640 1 -- 172.21.15.26:0/215313176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe95014cea0 con 0x7fe950163730 2024-04-26T21:08:45.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.384+0000 7f678f600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f67880cd520 con 0x7f678806eba0 2024-04-26T21:08:45.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.384+0000 7f678cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f678806e280 0x7f678814ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.384+0000 7f678cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f678806e280 0x7f678814ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.26:41872/0 (socket says 172.21.15.26:41872) 2024-04-26T21:08:45.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.384+0000 7f678cc00640 1 -- 172.21.15.26:0/123993643 learned_addr learned my addr 172.21.15.26:0/123993643 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:45.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.384+0000 7f678d600640 1 --2- 172.21.15.26:0/123993643 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67880b6c50 0x7f67881f0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.384+0000 7f678f600640 1 -- 172.21.15.26:0/123993643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f678806f050 con 0x7f678806e280 2024-04-26T21:08:45.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.384+0000 7f678f600640 1 -- 172.21.15.26:0/123993643 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f67880cd6a0 con 0x7f67880b6c50 2024-04-26T21:08:45.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.385+0000 7fd2a6600640 1 Processor -- start 2024-04-26T21:08:45.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.385+0000 7fc931200640 1 -- 172.21.15.26:0/1695282898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc92c06a700 con 0x7fc92c163730 2024-04-26T21:08:45.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.385+0000 7f88e3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f88dc145960 0x7f88dc0d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.385+0000 7f88e3400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88dc06fd20 0x7f88dc073060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.385+0000 7f88e3400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88dc145d40 con 0x7f88dc06f070 2024-04-26T21:08:45.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.385+0000 7f88e3400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f88dc0722a0 con 0x7f88dc145960 2024-04-26T21:08:45.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.385+0000 7f88e3400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f88dc072580 con 0x7f88dc06fd20 2024-04-26T21:08:45.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.385+0000 7fe956600640 1 -- 172.21.15.26:0/215313176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe950164930 con 0x7fe950163730 2024-04-26T21:08:45.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.385+0000 7f678cc00640 1 -- 172.21.15.26:0/123993643 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67880b6c50 msgr2=0x7f67881f0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.385+0000 7f678cc00640 1 --2- 172.21.15.26:0/123993643 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67880b6c50 0x7f67881f0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.386+0000 7f678cc00640 1 -- 172.21.15.26:0/123993643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678806eba0 msgr2=0x7f678814f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.386+0000 7f678cc00640 1 --2- 172.21.15.26:0/123993643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678806eba0 0x7f678814f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.385+0000 7fd2a6600640 1 -- start start 2024-04-26T21:08:45.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.386+0000 7f678cc00640 1 -- 172.21.15.26:0/123993643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678814fa30 con 0x7f678806e280 2024-04-26T21:08:45.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.386+0000 7f678d600640 1 --2- 172.21.15.26:0/123993643 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67880b6c50 0x7f67881f0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:08:45.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.386+0000 7f88e0a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f88dc145960 0x7f88dc0d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.386+0000 7f88e0a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f88dc145960 0x7f88dc0d02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.26:41876/0 (socket says 172.21.15.26:41876) 2024-04-26T21:08:45.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.386+0000 7f88e0a00640 1 -- 172.21.15.26:0/2903262949 learned_addr learned my addr 172.21.15.26:0/2903262949 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:45.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.386+0000 7fd2a6600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a0073790 0x7fd2a01449f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.386+0000 7fd2a6600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a00b5ea0 0x7fd2a00b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.386+0000 7f88e1400640 1 --2- 172.21.15.26:0/2903262949 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88dc06fd20 0x7f88dc073060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.387+0000 7f88e0a00640 1 -- 172.21.15.26:0/2903262949 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88dc06fd20 msgr2=0x7f88dc073060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:45.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.387+0000 7f88e0a00640 1 --2- 172.21.15.26:0/2903262949 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88dc06fd20 0x7f88dc073060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.387+0000 7f88e0a00640 1 -- 172.21.15.26:0/2903262949 >> v1:172.21.15.26:6789/0 conn(0x7f88dc06f070 legacy=0x7f88dc06f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:45.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.387+0000 7f88e0a00640 1 -- 172.21.15.26:0/2903262949 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88dc072860 con 0x7f88dc145960 2024-04-26T21:08:45.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.387+0000 7fd2a6600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd2a01522f0 0x7fd2a00b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.387+0000 7fd2a6600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a0070100 con 0x7fd2a00b5ea0 2024-04-26T21:08:45.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.387+0000 7fd2a6600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a00707f0 con 0x7fd2a0073790 2024-04-26T21:08:45.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.387+0000 7fd2a6600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a0070280 con 0x7fd2a01522f0 2024-04-26T21:08:45.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.387+0000 7fd29f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a0073790 0x7fd2a01449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.387+0000 7fd29f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a0073790 0x7fd2a01449f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.26:41878/0 (socket says 172.21.15.26:41878) 2024-04-26T21:08:45.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.387+0000 7fd29f400640 1 -- 172.21.15.26:0/2073073420 learned_addr learned my addr 172.21.15.26:0/2073073420 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:45.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.388+0000 7fd29fe00640 1 --2- 172.21.15.26:0/2073073420 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd2a01522f0 0x7fd2a00b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.388+0000 7fd29ea00640 1 --2- 172.21.15.26:0/2073073420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a00b5ea0 0x7fd2a00b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.388+0000 7f706be00640 1 --2- 172.21.15.26:0/1718190725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f706c0d1fd0 0x7f706c16a510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7054009870 tx=0x7f705402a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa50c814b1c067f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.388+0000 7fd29f400640 1 -- 172.21.15.26:0/2073073420 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd2a01522f0 msgr2=0x7fd2a00b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.388+0000 7fd29f400640 1 --2- 172.21.15.26:0/2073073420 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd2a01522f0 0x7fd2a00b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.388+0000 7fd29f400640 1 -- 172.21.15.26:0/2073073420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a00b5ea0 msgr2=0x7fd2a00b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.388+0000 7fd29f400640 1 --2- 172.21.15.26:0/2073073420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a00b5ea0 0x7fd2a00b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.389+0000 7fd29f400640 1 -- 172.21.15.26:0/2073073420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a0153510 con 0x7fd2a0073790 2024-04-26T21:08:45.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.389+0000 7fd29ea00640 1 --2- 172.21.15.26:0/2073073420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a00b5ea0 0x7fd2a00b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:08:45.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.389+0000 7fd29f400640 1 --2- 172.21.15.26:0/2073073420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a0073790 0x7fd2a01449f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd28802af90 tx=0x7fd2880046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.389+0000 7fe947400640 1 -- 172.21.15.26:0/215313176 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe93c018070 con 0x7fe950163730 2024-04-26T21:08:45.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.389+0000 7f678cc00640 1 --2- 172.21.15.26:0/123993643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f678806e280 0x7f678814ecc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f677c008d00 tx=0x7f677c00a540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.390+0000 7fe947400640 1 --2- 172.21.15.26:0/215313176 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe93006caf0 0x7fe93006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.390+0000 7fe947400640 1 -- 172.21.15.26:0/215313176 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe93c08b9f0 con 0x7fe950163730 2024-04-26T21:08:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.390+0000 7fc922000640 1 -- 172.21.15.26:0/1695282898 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc91802abd0 con 0x7fc92c163730 2024-04-26T21:08:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.391+0000 7f88e0a00640 1 --2- 172.21.15.26:0/2903262949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88dc145960 0x7f88dc0d02f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f88d000bb10 tx=0x7f88d002c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31378f41a85ea337 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.391+0000 7fe94f400640 1 --2- 172.21.15.26:0/215313176 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe93006caf0 0x7fe93006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.391+0000 7fc922000640 1 --2- 172.21.15.26:0/1695282898 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc90c06c9f0 0x7fc90c06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.391+0000 7fc922000640 1 -- 172.21.15.26:0/1695282898 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc9180ac710 con 0x7fc92c163730 2024-04-26T21:08:45.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.392+0000 7fc929e00640 1 --2- 172.21.15.26:0/1695282898 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc90c06c9f0 0x7fc90c06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.392+0000 7fe94f400640 1 --2- 172.21.15.26:0/215313176 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe93006caf0 0x7fe93006efb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe948033000 tx=0x7fe948009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.392+0000 7fc929e00640 1 --2- 172.21.15.26:0/1695282898 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc90c06c9f0 0x7fc90c06eeb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc91c009b60 tx=0x7fc91c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.397+0000 7fc922000640 1 -- 172.21.15.26:0/1695282898 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc92c1fa120 con 0x7fc92c163730 2024-04-26T21:08:45.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.403+0000 7fee0b200640 1 Processor -- start 2024-04-26T21:08:45.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.404+0000 7fee0b200640 1 -- start start 2024-04-26T21:08:45.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.404+0000 7f6784c00640 1 -- 172.21.15.26:0/123993643 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c01b070 con 0x7f678806e280 2024-04-26T21:08:45.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.404+0000 7f88db400640 1 -- 172.21.15.26:0/2903262949 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d003a070 con 0x7f88dc145960 2024-04-26T21:08:45.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.404+0000 7f706a000640 1 -- 172.21.15.26:0/1718190725 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054037070 con 0x7f706c0d1fd0 2024-04-26T21:08:45.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.405+0000 7fd297400640 1 -- 172.21.15.26:0/2073073420 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288047070 con 0x7fd2a0073790 2024-04-26T21:08:45.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.405+0000 7f88db400640 1 -- 172.21.15.26:0/2903262949 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88d00326f0 con 0x7f88dc145960 2024-04-26T21:08:45.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.404+0000 7f678f600640 1 -- 172.21.15.26:0/123993643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f678814fd10 con 0x7f678806e280 2024-04-26T21:08:45.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.405+0000 7f678f600640 1 -- 172.21.15.26:0/123993643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67881502d0 con 0x7f678806e280 2024-04-26T21:08:45.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.405+0000 7f6784c00640 1 -- 172.21.15.26:0/123993643 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f677c008df0 con 0x7f678806e280 2024-04-26T21:08:45.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.405+0000 7f6784c00640 1 -- 172.21.15.26:0/123993643 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c00b040 con 0x7f678806e280 2024-04-26T21:08:45.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.405+0000 7fee0b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fee0c0d1c80 0x7fee0c0d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.405+0000 7fd2a6600640 1 -- 172.21.15.26:0/2073073420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a01537f0 con 0x7fd2a0073790 2024-04-26T21:08:45.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.405+0000 7fd2a6600640 1 -- 172.21.15.26:0/2073073420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a0153db0 con 0x7fd2a0073790 2024-04-26T21:08:45.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.405+0000 7fe947400640 1 -- 172.21.15.26:0/215313176 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe95014cc20 con 0x7fe950163730 2024-04-26T21:08:45.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.405+0000 7f88db400640 1 -- 172.21.15.26:0/2903262949 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0032ac0 con 0x7f88dc145960 2024-04-26T21:08:45.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.405+0000 7fee0b200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fee0c162ba0 0x7fee0c0d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.405+0000 7fee0b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee0c0b61c0 con 0x7fee0c0d2060 2024-04-26T21:08:45.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.405+0000 7fee0b200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fee0c0b5f70 con 0x7fee0c162ba0 2024-04-26T21:08:45.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.405+0000 7fee0b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fee0c0d3830 con 0x7fee0c0d1c80 2024-04-26T21:08:45.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.405+0000 7f706a000640 1 -- 172.21.15.26:0/1718190725 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7054032640 con 0x7f706c0d1fd0 2024-04-26T21:08:45.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.405+0000 7f706a000640 1 -- 172.21.15.26:0/1718190725 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054037070 con 0x7f706c0d1fd0 2024-04-26T21:08:45.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.406+0000 7f7072000640 1 -- 172.21.15.26:0/1718190725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f706c0d1fd0 msgr2=0x7f706c16a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.406+0000 7f7072000640 1 --2- 172.21.15.26:0/1718190725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f706c0d1fd0 0x7f706c16a510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7054009870 tx=0x7f705402a330 comp rx=0 tx=0).stop 2024-04-26T21:08:45.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.406+0000 7f88e3400640 1 -- 172.21.15.26:0/2903262949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88dc145960 msgr2=0x7f88dc0d02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.406+0000 7fd297400640 1 -- 172.21.15.26:0/2073073420 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd288007640 con 0x7fd2a0073790 2024-04-26T21:08:45.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.406+0000 7fd297400640 1 -- 172.21.15.26:0/2073073420 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288038650 con 0x7fd2a0073790 2024-04-26T21:08:45.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.406+0000 7f88e3400640 1 --2- 172.21.15.26:0/2903262949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88dc145960 0x7f88dc0d02f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f88d000bb10 tx=0x7f88d002c830 comp rx=0 tx=0).stop 2024-04-26T21:08:45.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.406+0000 7fee09400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fee0c0d2060 0x7fee0c162490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.26:34050/0 (socket says 172.21.15.26:34050) 2024-04-26T21:08:45.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.406+0000 7fee09400640 1 -- 172.21.15.26:0/980973954 learned_addr learned my addr 172.21.15.26:0/980973954 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:45.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.406+0000 7f7072000640 1 -- 172.21.15.26:0/1718190725 shutdown_connections 2024-04-26T21:08:45.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.406+0000 7f7072000640 1 --2- 172.21.15.26:0/1718190725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f706c0d1fd0 0x7f706c16a510 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.406+0000 7f7072000640 1 -- 172.21.15.26:0/1718190725 >> 172.21.15.26:0/1718190725 conn(0x7f706c09e490 msgr2=0x7f706c0d51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.406+0000 7fee0a800640 1 --2- 172.21.15.26:0/980973954 >> v2:172.21.15.26:3300/0 conn(0x7fee0c162ba0 0x7fee0c0d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.407+0000 7f7072000640 1 -- 172.21.15.26:0/1718190725 shutdown_connections 2024-04-26T21:08:45.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.407+0000 7f88e3400640 1 -- 172.21.15.26:0/2903262949 shutdown_connections 2024-04-26T21:08:45.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.407+0000 7f88e3400640 1 --2- 172.21.15.26:0/2903262949 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88dc06fd20 0x7f88dc073060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.407+0000 7f88e3400640 1 --2- 172.21.15.26:0/2903262949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88dc145960 0x7f88dc0d02f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.407+0000 7f88e3400640 1 -- 172.21.15.26:0/2903262949 >> 172.21.15.26:0/2903262949 conn(0x7f88dc0b2af0 msgr2=0x7f88dc0b2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.407+0000 7f678f600640 1 -- 172.21.15.26:0/123993643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6764005290 con 0x7f678806e280 2024-04-26T21:08:45.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.407+0000 7f7072000640 1 -- 172.21.15.26:0/1718190725 wait complete. 2024-04-26T21:08:45.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.408+0000 7f88e3400640 1 -- 172.21.15.26:0/2903262949 shutdown_connections 2024-04-26T21:08:45.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.408+0000 7fd2a6600640 1 -- 172.21.15.26:0/2073073420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd278005290 con 0x7fd2a0073790 2024-04-26T21:08:45.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.409+0000 7f6784c00640 1 -- 172.21.15.26:0/123993643 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f677c01e070 con 0x7f678806e280 2024-04-26T21:08:45.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.410+0000 7f6784c00640 1 --2- 172.21.15.26:0/123993643 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f676806c920 0x7f676806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.407+0000 7fee0a800640 1 -- 172.21.15.26:0/980973954 >> v2:172.21.15.86:3300/0 conn(0x7fee0c0d1c80 msgr2=0x7fee0c0d3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:45.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.407+0000 7fee08a00640 1 -- 172.21.15.26:0/980973954 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2932993233 0 0) 0x7fee0c0b61c0 con 0x7fee0c0d2060 2024-04-26T21:08:45.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.407+0000 7fee0a800640 1 --2- 172.21.15.26:0/980973954 >> v2:172.21.15.86:3300/0 conn(0x7fee0c0d1c80 0x7fee0c0d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.409+0000 7fee0a800640 1 -- 172.21.15.26:0/980973954 >> v1:172.21.15.86:6789/0 conn(0x7fee0c0d2060 legacy=0x7fee0c162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.409+0000 7fee0a800640 1 -- 172.21.15.26:0/980973954 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0c0b64a0 con 0x7fee0c162ba0 2024-04-26T21:08:45.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.410+0000 7f6784c00640 1 -- 172.21.15.26:0/123993643 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f677c08b320 con 0x7f678806e280 2024-04-26T21:08:45.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.411+0000 7fee0a800640 1 --2- 172.21.15.26:0/980973954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee0c162ba0 0x7fee0c0d0cf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fee0c08d3b0 tx=0x7fee0402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212df8611d64145f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.410+0000 7f7072000640 1 Processor -- start 2024-04-26T21:08:45.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.409+0000 7f88e3400640 1 -- 172.21.15.26:0/2903262949 wait complete. 2024-04-26T21:08:45.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.411+0000 7f6787e00640 1 --2- 172.21.15.26:0/123993643 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f676806c920 0x7f676806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.411+0000 7f7072000640 1 -- start start 2024-04-26T21:08:45.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.410+0000 7f88e3400640 1 Processor -- start 2024-04-26T21:08:45.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.411+0000 7f88e3400640 1 -- start start 2024-04-26T21:08:45.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.411+0000 7fee08a00640 1 -- 172.21.15.26:0/980973954 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee04007740 con 0x7fee0c162ba0 2024-04-26T21:08:45.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.411+0000 7fee08a00640 1 -- 172.21.15.26:0/980973954 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee040078e0 con 0x7fee0c162ba0 2024-04-26T21:08:45.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.411+0000 7fee08a00640 1 -- 172.21.15.26:0/980973954 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee04007cf0 con 0x7fee0c162ba0 2024-04-26T21:08:45.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.412+0000 7f88e3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88dc06f070 0x7f88dc1552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.412+0000 7fd297400640 1 -- 172.21.15.26:0/2073073420 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd28804a070 con 0x7fd2a0073790 2024-04-26T21:08:45.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.412+0000 7f88e3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88dc06fd20 0x7f88dc13a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.412+0000 7f88e3400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88dc145960 0x7f88dc15ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.412+0000 7f88e3400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc0b4930 con 0x7f88dc06f070 2024-04-26T21:08:45.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.412+0000 7f88e3400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc0d0830 con 0x7f88dc06fd20 2024-04-26T21:08:45.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.412+0000 7f88e3400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f88dc0b4ab0 con 0x7f88dc145960 2024-04-26T21:08:45.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.413+0000 7fee0b200640 1 -- 172.21.15.26:0/980973954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee0c162ba0 msgr2=0x7fee0c0d0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.413+0000 7fee0b200640 1 --2- 172.21.15.26:0/980973954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee0c162ba0 0x7fee0c0d0cf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fee0c08d3b0 tx=0x7fee0402a610 comp rx=0 tx=0).stop 2024-04-26T21:08:45.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.413+0000 7f88e0a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88dc06f070 0x7f88dc1552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.413+0000 7f88e0a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88dc06f070 0x7f88dc1552b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34738/0 (socket says 172.21.15.26:34738) 2024-04-26T21:08:45.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.413+0000 7f88e0a00640 1 -- 172.21.15.26:0/993029483 learned_addr learned my addr 172.21.15.26:0/993029483 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:45.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.413+0000 7fd297400640 1 --2- 172.21.15.26:0/2073073420 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd27c06c8f0 0x7fd27c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.413+0000 7f88e1400640 1 --2- 172.21.15.26:0/993029483 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88dc145960 0x7f88dc15ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.413+0000 7f88dbe00640 1 --2- 172.21.15.26:0/993029483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88dc06fd20 0x7f88dc13a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.413+0000 7fd297400640 1 -- 172.21.15.26:0/2073073420 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd2880ac3b0 con 0x7fd2a0073790 2024-04-26T21:08:45.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.413+0000 7f88e0a00640 1 -- 172.21.15.26:0/993029483 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88dc145960 msgr2=0x7f88dc15ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.413+0000 7f88e0a00640 1 --2- 172.21.15.26:0/993029483 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88dc145960 0x7f88dc15ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.413+0000 7f6787e00640 1 --2- 172.21.15.26:0/123993643 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f676806c920 0x7f676806ede0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6774037000 tx=0x7f6774032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.413+0000 7f7072000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f706c0b4470 0x7f706c160f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.413+0000 7f7072000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f706c0cfd30 0x7f706c09f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.413+0000 7f7072000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f706c0d1fd0 0x7f706c0c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.414+0000 7fee0b200640 1 -- 172.21.15.26:0/980973954 shutdown_connections 2024-04-26T21:08:45.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.414+0000 7fee0b200640 1 --2- 172.21.15.26:0/980973954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee0c162ba0 0x7fee0c0d0cf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.414+0000 7fee0b200640 1 --2- 172.21.15.26:0/980973954 >> v2:172.21.15.86:3300/0 conn(0x7fee0c0d1c80 0x7fee0c0d3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.414+0000 7fee0b200640 1 -- 172.21.15.26:0/980973954 >> 172.21.15.26:0/980973954 conn(0x7fee0c0d5020 msgr2=0x7fee0c0d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.414+0000 7f7072000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f706c0d0b90 con 0x7f706c0d1fd0 2024-04-26T21:08:45.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.414+0000 7f7072000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f706c0d0f00 con 0x7f706c0b4470 2024-04-26T21:08:45.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.414+0000 7f7072000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f706c0d41d0 con 0x7f706c0cfd30 2024-04-26T21:08:45.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.414+0000 7fee0b200640 1 -- 172.21.15.26:0/980973954 shutdown_connections 2024-04-26T21:08:45.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.414+0000 7f88e0a00640 1 -- 172.21.15.26:0/993029483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88dc06fd20 msgr2=0x7f88dc13a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.414+0000 7f88e0a00640 1 --2- 172.21.15.26:0/993029483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88dc06fd20 0x7f88dc13a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.414+0000 7f88e0a00640 1 -- 172.21.15.26:0/993029483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88dc13b060 con 0x7f88dc06f070 2024-04-26T21:08:45.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.414+0000 7fd29ea00640 1 --2- 172.21.15.26:0/2073073420 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd27c06c8f0 0x7fd27c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.414+0000 7f88dbe00640 1 --2- 172.21.15.26:0/993029483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88dc06fd20 0x7f88dc13a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:08:45.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.414+0000 7f706aa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f706c0cfd30 0x7f706c09f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.414+0000 7f88e0a00640 1 --2- 172.21.15.26:0/993029483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88dc06f070 0x7f88dc1552b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f88d002c870 tx=0x7f88d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.414+0000 7fee0b200640 1 -- 172.21.15.26:0/980973954 wait complete. 2024-04-26T21:08:45.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.415+0000 7f706be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f706c0d1fd0 0x7f706c0c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.415+0000 7f706aa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f706c0cfd30 0x7f706c09f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.26:46116/0 (socket says 172.21.15.26:46116) 2024-04-26T21:08:45.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.415+0000 7f706aa00640 1 -- 172.21.15.26:0/95529072 learned_addr learned my addr 172.21.15.26:0/95529072 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:45.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.415+0000 7f706b400640 1 --2- 172.21.15.26:0/95529072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f706c0b4470 0x7f706c160f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.415+0000 7f88d8c00640 1 -- 172.21.15.26:0/993029483 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0045070 con 0x7f88dc06f070 2024-04-26T21:08:45.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.415+0000 7f88d8c00640 1 -- 172.21.15.26:0/993029483 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88d0035690 con 0x7f88dc06f070 2024-04-26T21:08:45.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.415+0000 7f88d8c00640 1 -- 172.21.15.26:0/993029483 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d003a070 con 0x7f88dc06f070 2024-04-26T21:08:45.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.415+0000 7f706be00640 1 -- 172.21.15.26:0/95529072 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f706c0cfd30 msgr2=0x7f706c09f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.416+0000 7f706be00640 1 --2- 172.21.15.26:0/95529072 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f706c0cfd30 0x7f706c09f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.416+0000 7f706be00640 1 -- 172.21.15.26:0/95529072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f706c0b4470 msgr2=0x7f706c160f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.416+0000 7f706be00640 1 --2- 172.21.15.26:0/95529072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f706c0b4470 0x7f706c160f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.416+0000 7fd29ea00640 1 --2- 172.21.15.26:0/2073073420 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd27c06c8f0 0x7fd27c06edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd2a015fe20 tx=0x7fd298009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.416+0000 7f706be00640 1 -- 172.21.15.26:0/95529072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f706c1614d0 con 0x7f706c0d1fd0 2024-04-26T21:08:45.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.416+0000 7f706aa00640 1 --2- 172.21.15.26:0/95529072 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f706c0cfd30 0x7f706c09f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:08:45.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.416+0000 7f706be00640 1 --2- 172.21.15.26:0/95529072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f706c0d1fd0 0x7f706c0c9d30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7054031000 tx=0x7f70540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.416+0000 7f88e3400640 1 -- 172.21.15.26:0/993029483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88dc13b340 con 0x7f88dc06f070 2024-04-26T21:08:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.416+0000 7f88e3400640 1 -- 172.21.15.26:0/993029483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88dc13b900 con 0x7f88dc06f070 2024-04-26T21:08:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.417+0000 7f7063400640 1 -- 172.21.15.26:0/95529072 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054041020 con 0x7f706c0d1fd0 2024-04-26T21:08:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.417+0000 7f7072000640 1 -- 172.21.15.26:0/95529072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f706c09fdf0 con 0x7f706c0d1fd0 2024-04-26T21:08:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.417+0000 7f7072000640 1 -- 172.21.15.26:0/95529072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f706c0a2840 con 0x7f706c0d1fd0 2024-04-26T21:08:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.417+0000 7fee0b200640 1 Processor -- start 2024-04-26T21:08:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.418+0000 7f7063400640 1 -- 172.21.15.26:0/95529072 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7054032640 con 0x7f706c0d1fd0 2024-04-26T21:08:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.418+0000 7f7063400640 1 -- 172.21.15.26:0/95529072 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054041020 con 0x7f706c0d1fd0 2024-04-26T21:08:45.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.418+0000 7fee0b200640 1 -- start start 2024-04-26T21:08:45.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.420+0000 7fee0b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee0c0d1c80 0x7fee0c150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.420+0000 7fee0b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee0c0d2060 0x7fee0c1517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.419+0000 7f7072000640 1 -- 172.21.15.26:0/95529072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7044005290 con 0x7f706c0d1fd0 2024-04-26T21:08:45.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.420+0000 7fee0b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee0c162ba0 0x7fee0c0ba140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.420+0000 7f88e3400640 1 -- 172.21.15.26:0/993029483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88dc06a700 con 0x7f88dc06f070 2024-04-26T21:08:45.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.421+0000 7fee0b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c0d5cc0 con 0x7fee0c162ba0 2024-04-26T21:08:45.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.421+0000 7fee0b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c0d5b40 con 0x7fee0c0d2060 2024-04-26T21:08:45.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.421+0000 7fee0a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee0c162ba0 0x7fee0c0ba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.421+0000 7fee0a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee0c162ba0 0x7fee0c0ba140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34752/0 (socket says 172.21.15.26:34752) 2024-04-26T21:08:45.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.421+0000 7fee0a800640 1 -- 172.21.15.26:0/570043445 learned_addr learned my addr 172.21.15.26:0/570043445 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:45.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.421+0000 7fee09e00640 1 --2- 172.21.15.26:0/570043445 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee0c0d1c80 0x7fee0c150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.421+0000 7fee09400640 1 --2- 172.21.15.26:0/570043445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee0c0d2060 0x7fee0c1517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.421+0000 7fee0b200640 1 -- 172.21.15.26:0/570043445 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fee0c0c8b70 con 0x7fee0c0d1c80 2024-04-26T21:08:45.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.421+0000 7f88d8c00640 1 -- 172.21.15.26:0/993029483 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f88d0048070 con 0x7f88dc06f070 2024-04-26T21:08:45.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.422+0000 7fee0a800640 1 -- 172.21.15.26:0/570043445 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee0c0d1c80 msgr2=0x7fee0c150570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.422+0000 7fee0a800640 1 --2- 172.21.15.26:0/570043445 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee0c0d1c80 0x7fee0c150570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.422+0000 7fee0a800640 1 -- 172.21.15.26:0/570043445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee0c0d2060 msgr2=0x7fee0c1517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.422+0000 7fee0a800640 1 --2- 172.21.15.26:0/570043445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee0c0d2060 0x7fee0c1517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.422+0000 7fee0a800640 1 -- 172.21.15.26:0/570043445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0c0cc190 con 0x7fee0c162ba0 2024-04-26T21:08:45.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.422+0000 7fee0a800640 1 --2- 172.21.15.26:0/570043445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee0c162ba0 0x7fee0c0ba140 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fee04009e80 tx=0x7fee04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.422+0000 7f7063400640 1 -- 172.21.15.26:0/95529072 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7054044020 con 0x7f706c0d1fd0 2024-04-26T21:08:45.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.422+0000 7f88d8c00640 1 --2- 172.21.15.26:0/993029483 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f88bc06c970 0x7f88bc06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.424+0000 7f7063400640 1 --2- 172.21.15.26:0/95529072 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f704806caf0 0x7f704806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.424+0000 7f7063400640 1 -- 172.21.15.26:0/95529072 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f70540ac150 con 0x7f706c0d1fd0 2024-04-26T21:08:45.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.424+0000 7f88d8c00640 1 -- 172.21.15.26:0/993029483 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f88d00abc90 con 0x7f88dc06f070 2024-04-26T21:08:45.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.424+0000 7f706b400640 1 --2- 172.21.15.26:0/95529072 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f704806caf0 0x7f704806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.425+0000 7fd297400640 1 -- 172.21.15.26:0/2073073420 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd28803f070 con 0x7fd2a0073790 2024-04-26T21:08:45.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.425+0000 7f88dbe00640 1 --2- 172.21.15.26:0/993029483 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f88bc06c970 0x7f88bc06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.426+0000 7f706b400640 1 --2- 172.21.15.26:0/95529072 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f704806caf0 0x7f704806efb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7064007770 tx=0x7f7064009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.426+0000 7f88dbe00640 1 --2- 172.21.15.26:0/993029483 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f88bc06c970 0x7f88bc06ee30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f88dc08cf40 tx=0x7f88c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.429+0000 7fee02000640 1 -- 172.21.15.26:0/570043445 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0403f070 con 0x7fee0c162ba0 2024-04-26T21:08:45.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.430+0000 7f88d8c00640 1 -- 172.21.15.26:0/993029483 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f88d0079b50 con 0x7f88dc06f070 2024-04-26T21:08:45.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.430+0000 7fee0b200640 1 -- 172.21.15.26:0/570043445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee0c0cc430 con 0x7fee0c162ba0 2024-04-26T21:08:45.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.430+0000 7fee0b200640 1 -- 172.21.15.26:0/570043445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee0c0cc9f0 con 0x7fee0c162ba0 2024-04-26T21:08:45.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.431+0000 7fee0b200640 1 -- 172.21.15.26:0/570043445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee0c06a700 con 0x7fee0c162ba0 2024-04-26T21:08:45.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.439+0000 7f6784c00640 1 -- 172.21.15.26:0/123993643 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f677c0590f0 con 0x7f678806e280 2024-04-26T21:08:45.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.439+0000 7fee02000640 1 -- 172.21.15.26:0/570043445 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee04007850 con 0x7fee0c162ba0 2024-04-26T21:08:45.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.440+0000 7fee02000640 1 -- 172.21.15.26:0/570043445 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee04031e10 con 0x7fee0c162ba0 2024-04-26T21:08:45.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.438+0000 7f7063400640 1 -- 172.21.15.26:0/95529072 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f705404b030 con 0x7f706c0d1fd0 2024-04-26T21:08:45.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.442+0000 7fee02000640 1 -- 172.21.15.26:0/570043445 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee0403b410 con 0x7fee0c162ba0 2024-04-26T21:08:45.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.444+0000 7fee02000640 1 --2- 172.21.15.26:0/570043445 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fedec06c8f0 0x7fedec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:45.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.444+0000 7fee02000640 1 -- 172.21.15.26:0/570043445 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fee040ad6c0 con 0x7fee0c162ba0 2024-04-26T21:08:45.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.445+0000 7fee09e00640 1 --2- 172.21.15.26:0/570043445 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fedec06c8f0 0x7fedec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:45.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.447+0000 7fee09e00640 1 --2- 172.21.15.26:0/570043445 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fedec06c8f0 0x7fedec06edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0098e0 tx=0x7fedfc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:45.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.447+0000 7fee02000640 1 -- 172.21.15.26:0/570043445 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fee0407ba50 con 0x7fee0c162ba0 2024-04-26T21:08:45.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.811+0000 7f145b400640 1 -- 172.21.15.26:0/715308174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1454151770 con 0x7f14540cf960 2024-04-26T21:08:45.813 INFO:teuthology.orchestra.run.smithi026.stdout:55834574851 2024-04-26T21:08:45.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.812+0000 7f1450c00640 1 -- 172.21.15.26:0/715308174 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1440059520 con 0x7f14540cf960 2024-04-26T21:08:45.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.817+0000 7f144a000640 1 -- 172.21.15.26:0/715308174 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f143406c9c0 msgr2=0x7f143406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.817+0000 7f144a000640 1 --2- 172.21.15.26:0/715308174 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f143406c9c0 0x7f143406ee80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f144c005c00 tx=0x7f144c009040 comp rx=0 tx=0).stop 2024-04-26T21:08:45.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.817+0000 7f144a000640 1 -- 172.21.15.26:0/715308174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14540cf960 msgr2=0x7f14541e44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.817+0000 7f144a000640 1 --2- 172.21.15.26:0/715308174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14540cf960 0x7f14541e44b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f144000b440 tx=0x7f144000b900 comp rx=0 tx=0).stop 2024-04-26T21:08:45.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.818+0000 7f144a000640 1 -- 172.21.15.26:0/715308174 shutdown_connections 2024-04-26T21:08:45.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.818+0000 7f144a000640 1 --2- 172.21.15.26:0/715308174 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f143406c9c0 0x7f143406ee80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.818+0000 7f144a000640 1 --2- 172.21.15.26:0/715308174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14540cf960 0x7f14541e44b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.818+0000 7f144a000640 1 --2- 172.21.15.26:0/715308174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14540b6c50 0x7f1454155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.818+0000 7f144a000640 1 --2- 172.21.15.26:0/715308174 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f145406e280 0x7f1454147280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.818+0000 7f144a000640 1 -- 172.21.15.26:0/715308174 >> 172.21.15.26:0/715308174 conn(0x7f14540b2af0 msgr2=0x7f1454072320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.818+0000 7fc931200640 1 -- 172.21.15.26:0/1695282898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc92c0cf000 con 0x7fc92c163730 2024-04-26T21:08:45.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.819+0000 7fc922000640 1 -- 172.21.15.26:0/1695282898 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc91807a560 con 0x7fc92c163730 2024-04-26T21:08:45.820 INFO:teuthology.orchestra.run.smithi026.stdout:55834574853 2024-04-26T21:08:45.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.819+0000 7f144a000640 1 -- 172.21.15.26:0/715308174 shutdown_connections 2024-04-26T21:08:45.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.820+0000 7f144a000640 1 -- 172.21.15.26:0/715308174 wait complete. 2024-04-26T21:08:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.824+0000 7fc931200640 1 -- 172.21.15.26:0/1695282898 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc90c06c9f0 msgr2=0x7fc90c06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.824+0000 7fc931200640 1 --2- 172.21.15.26:0/1695282898 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc90c06c9f0 0x7fc90c06eeb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc91c009b60 tx=0x7fc91c009190 comp rx=0 tx=0).stop 2024-04-26T21:08:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.824+0000 7fc931200640 1 -- 172.21.15.26:0/1695282898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc92c163730 msgr2=0x7fc92c0b2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.824+0000 7fc931200640 1 --2- 172.21.15.26:0/1695282898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc92c163730 0x7fc92c0b2010 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc92c06e450 tx=0x7fc9180046c0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.824+0000 7fc931200640 1 -- 172.21.15.26:0/1695282898 shutdown_connections 2024-04-26T21:08:45.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.824+0000 7fc931200640 1 --2- 172.21.15.26:0/1695282898 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc90c06c9f0 0x7fc90c06eeb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.824+0000 7fc931200640 1 --2- 172.21.15.26:0/1695282898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc92c163730 0x7fc92c0b2010 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.824+0000 7fc931200640 1 --2- 172.21.15.26:0/1695282898 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc92c0d13e0 0x7fc92c0b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.824+0000 7fc931200640 1 --2- 172.21.15.26:0/1695282898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc92c0b3df0 0x7fc92c1f9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.824+0000 7fc931200640 1 -- 172.21.15.26:0/1695282898 >> 172.21.15.26:0/1695282898 conn(0x7fc92c0cd840 msgr2=0x7fc92c0cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.825+0000 7fc931200640 1 -- 172.21.15.26:0/1695282898 shutdown_connections 2024-04-26T21:08:45.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.825+0000 7fc931200640 1 -- 172.21.15.26:0/1695282898 wait complete. 2024-04-26T21:08:45.849 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.0 2024-04-26T21:08:45.852 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-04-26T21:08:45.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.859+0000 7f88e3400640 1 -- 172.21.15.26:0/993029483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f88dc06a700 con 0x7f88dc06f070 2024-04-26T21:08:45.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.860+0000 7f88d8c00640 1 -- 172.21.15.26:0/993029483 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f88d007de50 con 0x7f88dc06f070 2024-04-26T21:08:45.860 INFO:teuthology.orchestra.run.smithi026.stdout:55834574851 2024-04-26T21:08:45.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.861+0000 7fe956600640 1 -- 172.21.15.26:0/215313176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe95006a700 con 0x7fe950163730 2024-04-26T21:08:45.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.862+0000 7f88e3400640 1 -- 172.21.15.26:0/993029483 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f88bc06c970 msgr2=0x7f88bc06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.862+0000 7f88e3400640 1 --2- 172.21.15.26:0/993029483 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f88bc06c970 0x7f88bc06ee30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f88dc08cf40 tx=0x7f88c8009040 comp rx=0 tx=0).stop 2024-04-26T21:08:45.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.862+0000 7f88e3400640 1 -- 172.21.15.26:0/993029483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88dc06f070 msgr2=0x7f88dc1552b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.862+0000 7f88e3400640 1 --2- 172.21.15.26:0/993029483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88dc06f070 0x7f88dc1552b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f88d002c870 tx=0x7f88d0004730 comp rx=0 tx=0).stop 2024-04-26T21:08:45.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.862+0000 7f88e3400640 1 -- 172.21.15.26:0/993029483 shutdown_connections 2024-04-26T21:08:45.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.862+0000 7f88e3400640 1 --2- 172.21.15.26:0/993029483 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f88bc06c970 0x7f88bc06ee30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.862+0000 7f88e3400640 1 --2- 172.21.15.26:0/993029483 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88dc145960 0x7f88dc15ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.862+0000 7f88e3400640 1 --2- 172.21.15.26:0/993029483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88dc06fd20 0x7f88dc13a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.862+0000 7f88e3400640 1 --2- 172.21.15.26:0/993029483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88dc06f070 0x7f88dc1552b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.862+0000 7f88e3400640 1 -- 172.21.15.26:0/993029483 >> 172.21.15.26:0/993029483 conn(0x7f88dc0b2af0 msgr2=0x7f88dc06f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.863+0000 7f88e3400640 1 -- 172.21.15.26:0/993029483 shutdown_connections 2024-04-26T21:08:45.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.863+0000 7f88e3400640 1 -- 172.21.15.26:0/993029483 wait complete. 2024-04-26T21:08:45.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.863+0000 7fe947400640 1 -- 172.21.15.26:0/215313176 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe93c0600d0 con 0x7fe950163730 2024-04-26T21:08:45.865 INFO:teuthology.orchestra.run.smithi026.stdout:55834574851 2024-04-26T21:08:45.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.867+0000 7fe956600640 1 -- 172.21.15.26:0/215313176 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe93006caf0 msgr2=0x7fe93006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.867+0000 7fe956600640 1 --2- 172.21.15.26:0/215313176 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe93006caf0 0x7fe93006efb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe948033000 tx=0x7fe948009000 comp rx=0 tx=0).stop 2024-04-26T21:08:45.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.867+0000 7fe956600640 1 -- 172.21.15.26:0/215313176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe950163730 msgr2=0x7fe950163b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.867+0000 7fe956600640 1 --2- 172.21.15.26:0/215313176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe950163730 0x7fe950163b10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00c7f0 tx=0x7fe93c00ccb0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.867+0000 7fe956600640 1 -- 172.21.15.26:0/215313176 shutdown_connections 2024-04-26T21:08:45.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.867+0000 7fe956600640 1 --2- 172.21.15.26:0/215313176 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe93006caf0 0x7fe93006efb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.867+0000 7fe956600640 1 --2- 172.21.15.26:0/215313176 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe950167180 0x7fe9500722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.867+0000 7fe956600640 1 --2- 172.21.15.26:0/215313176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe950163730 0x7fe950163b10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.867+0000 7fe956600640 1 --2- 172.21.15.26:0/215313176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9500b3df0 0x7fe95014d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.867+0000 7fe956600640 1 -- 172.21.15.26:0/215313176 >> 172.21.15.26:0/215313176 conn(0x7fe9500cd840 msgr2=0x7fe9500cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.868+0000 7fe956600640 1 -- 172.21.15.26:0/215313176 shutdown_connections 2024-04-26T21:08:45.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.868+0000 7fe956600640 1 -- 172.21.15.26:0/215313176 wait complete. 2024-04-26T21:08:45.881 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.5 2024-04-26T21:08:45.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.880+0000 7fd2a6600640 1 -- 172.21.15.26:0/2073073420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fd278005850 con 0x7fd2a0073790 2024-04-26T21:08:45.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.880+0000 7fd297400640 1 -- 172.21.15.26:0/2073073420 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd28807a230 con 0x7fd2a0073790 2024-04-26T21:08:45.881 INFO:teuthology.orchestra.run.smithi026.stdout:55834574851 2024-04-26T21:08:45.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.883+0000 7fd2a6600640 1 -- 172.21.15.26:0/2073073420 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd27c06c8f0 msgr2=0x7fd27c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.884+0000 7fd2a6600640 1 --2- 172.21.15.26:0/2073073420 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd27c06c8f0 0x7fd27c06edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd2a015fe20 tx=0x7fd298009040 comp rx=0 tx=0).stop 2024-04-26T21:08:45.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.884+0000 7fd2a6600640 1 -- 172.21.15.26:0/2073073420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a0073790 msgr2=0x7fd2a01449f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.884+0000 7fd2a6600640 1 --2- 172.21.15.26:0/2073073420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a0073790 0x7fd2a01449f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd28802af90 tx=0x7fd2880046c0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.884+0000 7fd2a6600640 1 -- 172.21.15.26:0/2073073420 shutdown_connections 2024-04-26T21:08:45.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.884+0000 7fd2a6600640 1 --2- 172.21.15.26:0/2073073420 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd27c06c8f0 0x7fd27c06edb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.884+0000 7fd2a6600640 1 --2- 172.21.15.26:0/2073073420 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd2a01522f0 0x7fd2a00b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.884+0000 7fd2a6600640 1 --2- 172.21.15.26:0/2073073420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a00b5ea0 0x7fd2a00b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.884+0000 7fd2a6600640 1 --2- 172.21.15.26:0/2073073420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a0073790 0x7fd2a01449f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.884+0000 7fd2a6600640 1 -- 172.21.15.26:0/2073073420 >> 172.21.15.26:0/2073073420 conn(0x7fd2a006ca30 msgr2=0x7fd2a0072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.884+0000 7fd2a6600640 1 -- 172.21.15.26:0/2073073420 shutdown_connections 2024-04-26T21:08:45.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.884+0000 7fd2a6600640 1 -- 172.21.15.26:0/2073073420 wait complete. 2024-04-26T21:08:45.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.887+0000 7fee0b200640 1 -- 172.21.15.26:0/570043445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fee0c0cf600 con 0x7fee0c162ba0 2024-04-26T21:08:45.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.887+0000 7fee02000640 1 -- 172.21.15.26:0/570043445 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fee0407b350 con 0x7fee0c162ba0 2024-04-26T21:08:45.888 INFO:teuthology.orchestra.run.smithi026.stdout:55834574851 2024-04-26T21:08:45.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.890+0000 7fee0b200640 1 -- 172.21.15.26:0/570043445 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fedec06c8f0 msgr2=0x7fedec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.890+0000 7fee0b200640 1 --2- 172.21.15.26:0/570043445 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fedec06c8f0 0x7fedec06edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0098e0 tx=0x7fedfc008040 comp rx=0 tx=0).stop 2024-04-26T21:08:45.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.890+0000 7fee0b200640 1 -- 172.21.15.26:0/570043445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee0c162ba0 msgr2=0x7fee0c0ba140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.890+0000 7fee0b200640 1 --2- 172.21.15.26:0/570043445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee0c162ba0 0x7fee0c0ba140 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fee04009e80 tx=0x7fee04004a40 comp rx=0 tx=0).stop 2024-04-26T21:08:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.890+0000 7fee0b200640 1 -- 172.21.15.26:0/570043445 shutdown_connections 2024-04-26T21:08:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.890+0000 7fee0b200640 1 --2- 172.21.15.26:0/570043445 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fedec06c8f0 0x7fedec06edb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.890+0000 7fee0b200640 1 --2- 172.21.15.26:0/570043445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee0c162ba0 0x7fee0c0ba140 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.890+0000 7fee0b200640 1 --2- 172.21.15.26:0/570043445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee0c0d2060 0x7fee0c1517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.890+0000 7fee0b200640 1 --2- 172.21.15.26:0/570043445 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee0c0d1c80 0x7fee0c150570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.890+0000 7fee0b200640 1 -- 172.21.15.26:0/570043445 >> 172.21.15.26:0/570043445 conn(0x7fee0c0d5020 msgr2=0x7fee0c15f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.892 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.2 2024-04-26T21:08:45.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.890+0000 7fee0b200640 1 -- 172.21.15.26:0/570043445 shutdown_connections 2024-04-26T21:08:45.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.890+0000 7fee0b200640 1 -- 172.21.15.26:0/570043445 wait complete. 2024-04-26T21:08:45.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.900+0000 7f678f600640 1 -- 172.21.15.26:0/123993643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f6764005580 con 0x7f678806e280 2024-04-26T21:08:45.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.901+0000 7f6784c00640 1 -- 172.21.15.26:0/123993643 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f677c05d3f0 con 0x7f678806e280 2024-04-26T21:08:45.902 INFO:teuthology.orchestra.run.smithi026.stdout:55834574851 2024-04-26T21:08:45.904 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.3 2024-04-26T21:08:45.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.904+0000 7f678f600640 1 -- 172.21.15.26:0/123993643 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f676806c920 msgr2=0x7f676806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.904+0000 7f678f600640 1 --2- 172.21.15.26:0/123993643 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f676806c920 0x7f676806ede0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6774037000 tx=0x7f6774032000 comp rx=0 tx=0).stop 2024-04-26T21:08:45.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.904+0000 7f678f600640 1 -- 172.21.15.26:0/123993643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f678806e280 msgr2=0x7f678814ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.904+0000 7f678f600640 1 --2- 172.21.15.26:0/123993643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f678806e280 0x7f678814ecc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f677c008d00 tx=0x7f677c00a540 comp rx=0 tx=0).stop 2024-04-26T21:08:45.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.904+0000 7f678f600640 1 -- 172.21.15.26:0/123993643 shutdown_connections 2024-04-26T21:08:45.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.904+0000 7f678f600640 1 --2- 172.21.15.26:0/123993643 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f676806c920 0x7f676806ede0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.904+0000 7f678f600640 1 --2- 172.21.15.26:0/123993643 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67880b6c50 0x7f67881f0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.904+0000 7f678f600640 1 --2- 172.21.15.26:0/123993643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678806eba0 0x7f678814f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.904+0000 7f678f600640 1 --2- 172.21.15.26:0/123993643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f678806e280 0x7f678814ecc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.904+0000 7f678f600640 1 -- 172.21.15.26:0/123993643 >> 172.21.15.26:0/123993643 conn(0x7f67880b2af0 msgr2=0x7f6788147930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.904+0000 7f678f600640 1 -- 172.21.15.26:0/123993643 shutdown_connections 2024-04-26T21:08:45.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.904+0000 7f678f600640 1 -- 172.21.15.26:0/123993643 wait complete. 2024-04-26T21:08:45.909 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.6 2024-04-26T21:08:45.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.914+0000 7f7072000640 1 -- 172.21.15.26:0/95529072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7044005580 con 0x7f706c0d1fd0 2024-04-26T21:08:45.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.914+0000 7f7063400640 1 -- 172.21.15.26:0/95529072 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f705407a4e0 con 0x7f706c0d1fd0 2024-04-26T21:08:45.915 INFO:teuthology.orchestra.run.smithi026.stdout:55834574852 2024-04-26T21:08:45.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.917+0000 7f7072000640 1 -- 172.21.15.26:0/95529072 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f704806caf0 msgr2=0x7f704806efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.917+0000 7f7072000640 1 --2- 172.21.15.26:0/95529072 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f704806caf0 0x7f704806efb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7064007770 tx=0x7f7064009040 comp rx=0 tx=0).stop 2024-04-26T21:08:45.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.917+0000 7f7072000640 1 -- 172.21.15.26:0/95529072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f706c0d1fd0 msgr2=0x7f706c0c9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:45.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.917+0000 7f7072000640 1 --2- 172.21.15.26:0/95529072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f706c0d1fd0 0x7f706c0c9d30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7054031000 tx=0x7f70540046c0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.917+0000 7f7072000640 1 -- 172.21.15.26:0/95529072 shutdown_connections 2024-04-26T21:08:45.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.917+0000 7f7072000640 1 --2- 172.21.15.26:0/95529072 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f704806caf0 0x7f704806efb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.917+0000 7f7072000640 1 --2- 172.21.15.26:0/95529072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f706c0d1fd0 0x7f706c0c9d30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.917+0000 7f7072000640 1 --2- 172.21.15.26:0/95529072 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f706c0cfd30 0x7f706c09f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.917+0000 7f7072000640 1 --2- 172.21.15.26:0/95529072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f706c0b4470 0x7f706c160f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:45.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.917+0000 7f7072000640 1 -- 172.21.15.26:0/95529072 >> 172.21.15.26:0/95529072 conn(0x7f706c09e490 msgr2=0x7f706c0b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:45.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.918+0000 7f7072000640 1 -- 172.21.15.26:0/95529072 shutdown_connections 2024-04-26T21:08:45.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:45.918+0000 7f7072000640 1 -- 172.21.15.26:0/95529072 wait complete. 2024-04-26T21:08:45.926 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.4 2024-04-26T21:08:45.936 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.1 2024-04-26T21:08:46.850 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T21:08:46.853 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T21:08:46.881 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T21:08:46.893 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T21:08:46.904 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T21:08:46.910 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T21:08:46.926 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T21:08:46.937 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T21:08:47.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.105+0000 7f0608a00640 1 Processor -- start 2024-04-26T21:08:47.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.105+0000 7f0608a00640 1 -- start start 2024-04-26T21:08:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.107+0000 7f0608a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f05fc00aad0 0x7f05fc0df070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.108+0000 7f0608a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f05fc050180 0x7f05fc0e0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.108+0000 7f0601e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f05fc0530a0 0x7f05fc04fa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44834/0 (socket says 172.21.15.26:44834) 2024-04-26T21:08:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.108+0000 7f0601e00640 1 -- 172.21.15.26:0/1733423 learned_addr learned my addr 172.21.15.26:0/1733423 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.108+0000 7f0603200640 1 --2- 172.21.15.26:0/1733423 >> v2:172.21.15.26:3300/0 conn(0x7f05fc050180 0x7f05fc0e0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.109+0000 7f0608a00640 1 -- 172.21.15.26:0/1733423 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05fc0535f0 con 0x7f05fc0530a0 2024-04-26T21:08:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.109+0000 7f0608a00640 1 -- 172.21.15.26:0/1733423 --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f05fc0538d0 con 0x7f05fc050180 2024-04-26T21:08:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.109+0000 7f0602800640 1 --2- 172.21.15.26:0/1733423 >> v2:172.21.15.86:3300/0 conn(0x7f05fc00aad0 0x7f05fc0df070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.109+0000 7f0601400640 1 -- 172.21.15.26:0/1733423 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2521219836 0 0) 0x7f05fc0535f0 con 0x7f05fc0530a0 2024-04-26T21:08:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.110+0000 7f0608a00640 1 -- 172.21.15.26:0/1733423 --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f05fc053bb0 con 0x7f05fc00aad0 2024-04-26T21:08:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.110+0000 7f0601400640 1 -- 172.21.15.26:0/1733423 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05e8003420 con 0x7f05fc0530a0 2024-04-26T21:08:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.110+0000 7f0603200640 1 -- 172.21.15.26:0/1733423 >> v2:172.21.15.86:3300/0 conn(0x7f05fc00aad0 msgr2=0x7f05fc0df070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.110+0000 7f0603200640 1 --2- 172.21.15.26:0/1733423 >> v2:172.21.15.86:3300/0 conn(0x7f05fc00aad0 0x7f05fc0df070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.110+0000 7f0603200640 1 -- 172.21.15.26:0/1733423 >> v1:172.21.15.26:6789/0 conn(0x7f05fc0530a0 legacy=0x7f05fc04fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.110+0000 7f0603200640 1 -- 172.21.15.26:0/1733423 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05fc053e90 con 0x7f05fc050180 2024-04-26T21:08:47.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.112+0000 7f0603200640 1 --2- 172.21.15.26:0/1733423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc050180 0x7f05fc0e0100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f05fc052790 tx=0x7f05f402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fa23179a45f325 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.113+0000 7f0601400640 1 -- 172.21.15.26:0/1733423 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4037070 con 0x7f05fc050180 2024-04-26T21:08:47.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.113+0000 7f0601400640 1 -- 172.21.15.26:0/1733423 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05f4032440 con 0x7f05fc050180 2024-04-26T21:08:47.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.113+0000 7f0601400640 1 -- 172.21.15.26:0/1733423 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4032850 con 0x7f05fc050180 2024-04-26T21:08:47.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.114+0000 7f0608a00640 1 -- 172.21.15.26:0/1733423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc050180 msgr2=0x7f05fc0e0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.114+0000 7f0608a00640 1 --2- 172.21.15.26:0/1733423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc050180 0x7f05fc0e0100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f05fc052790 tx=0x7f05f402a250 comp rx=0 tx=0).stop 2024-04-26T21:08:47.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.115+0000 7f0608a00640 1 -- 172.21.15.26:0/1733423 shutdown_connections 2024-04-26T21:08:47.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.115+0000 7f0608a00640 1 --2- 172.21.15.26:0/1733423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc050180 0x7f05fc0e0100 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.115+0000 7f0608a00640 1 --2- 172.21.15.26:0/1733423 >> v2:172.21.15.86:3300/0 conn(0x7f05fc00aad0 0x7f05fc0df070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.115+0000 7f0608a00640 1 -- 172.21.15.26:0/1733423 >> 172.21.15.26:0/1733423 conn(0x7f05fc009fb0 msgr2=0x7f05fc00a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:47.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.116+0000 7f0608a00640 1 -- 172.21.15.26:0/1733423 shutdown_connections 2024-04-26T21:08:47.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.116+0000 7f0608a00640 1 -- 172.21.15.26:0/1733423 wait complete. 2024-04-26T21:08:47.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.119+0000 7f0608a00640 1 Processor -- start 2024-04-26T21:08:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.119+0000 7f0608a00640 1 -- start start 2024-04-26T21:08:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.119+0000 7ff9ee600640 1 Processor -- start 2024-04-26T21:08:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.119+0000 7ff9ee600640 1 -- start start 2024-04-26T21:08:47.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.122+0000 7f0608a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05fc00aad0 0x7f05fc04b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.122+0000 7f0608a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc050180 0x7f05fc04e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.122+0000 7ff9ee600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff9e0109110 0x7ff9e0053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.123+0000 7ff9ee600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9e0109a90 con 0x7ff9e0108ca0 2024-04-26T21:08:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.123+0000 7ff9ee600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9e00071c0 con 0x7ff9e0105160 2024-04-26T21:08:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.123+0000 7ff9ee600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff9e003aaa0 con 0x7ff9e0109110 2024-04-26T21:08:47.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.122+0000 7f0602800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05fc00aad0 0x7f05fc04b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.123+0000 7f0601e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc050180 0x7f05fc04e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.123+0000 7f0601e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc050180 0x7f05fc04e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34770/0 (socket says 172.21.15.26:34770) 2024-04-26T21:08:47.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.123+0000 7f0602800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05fc00aad0 0x7f05fc04b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.26:41924/0 (socket says 172.21.15.26:41924) 2024-04-26T21:08:47.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.123+0000 7f0601e00640 1 -- 172.21.15.26:0/2832832722 learned_addr learned my addr 172.21.15.26:0/2832832722 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:47.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.123+0000 7f0608a00640 1 --2- 172.21.15.26:0/2832832722 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05fc0530a0 0x7f05fc0cee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.123+0000 7ff9e7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff9e0105160 0x7ff9e003a390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.26:34064/0 (socket says 172.21.15.26:34064) 2024-04-26T21:08:47.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.123+0000 7ff9e7400640 1 -- 172.21.15.26:0/1468232881 learned_addr learned my addr 172.21.15.26:0/1468232881 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:47.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.123+0000 7f0608a00640 1 -- 172.21.15.26:0/2832832722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc006160 con 0x7f05fc050180 2024-04-26T21:08:47.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.123+0000 7f0608a00640 1 -- 172.21.15.26:0/2832832722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc005fe0 con 0x7f05fc00aad0 2024-04-26T21:08:47.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.123+0000 7f0608a00640 1 -- 172.21.15.26:0/2832832722 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f05fc0062e0 con 0x7f05fc0530a0 2024-04-26T21:08:47.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.124+0000 7ff9e7e00640 1 --2- 172.21.15.26:0/1468232881 >> v2:172.21.15.86:3300/0 conn(0x7ff9e0109110 0x7ff9e0053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.124+0000 7f0603200640 1 --2- 172.21.15.26:0/2832832722 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05fc0530a0 0x7f05fc0cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.124+0000 7ff9e6000640 1 -- 172.21.15.26:0/1468232881 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1668950655 0 0) 0x7ff9e00071c0 con 0x7ff9e0105160 2024-04-26T21:08:47.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.124+0000 7ff9e6000640 1 -- 172.21.15.26:0/1468232881 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9d0003520 con 0x7ff9e0105160 2024-04-26T21:08:47.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.125+0000 7ff9e7e00640 1 -- 172.21.15.26:0/1468232881 >> v1:172.21.15.86:6789/0 conn(0x7ff9e0105160 legacy=0x7ff9e003a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.125+0000 7ff9e7e00640 1 -- 172.21.15.26:0/1468232881 >> v1:172.21.15.26:6789/0 conn(0x7ff9e0108ca0 legacy=0x7ff9e00534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.125+0000 7ff9e7e00640 1 -- 172.21.15.26:0/1468232881 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e00074a0 con 0x7ff9e0109110 2024-04-26T21:08:47.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.124+0000 7f0603200640 1 -- 172.21.15.26:0/2832832722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05fc00aad0 msgr2=0x7f05fc04b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.125+0000 7f0603200640 1 --2- 172.21.15.26:0/2832832722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05fc00aad0 0x7f05fc04b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.125+0000 7f0603200640 1 -- 172.21.15.26:0/2832832722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc050180 msgr2=0x7f05fc04e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.125+0000 7f0603200640 1 --2- 172.21.15.26:0/2832832722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc050180 0x7f05fc04e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.125+0000 7f0603200640 1 -- 172.21.15.26:0/2832832722 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05fc04d310 con 0x7f05fc0530a0 2024-04-26T21:08:47.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.125+0000 7f0602800640 1 --2- 172.21.15.26:0/2832832722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05fc00aad0 0x7f05fc04b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:47.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.126+0000 7f0603200640 1 --2- 172.21.15.26:0/2832832722 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05fc0530a0 0x7f05fc0cee50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f05f4031000 tx=0x7f05f40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.126+0000 7ff9e7e00640 1 --2- 172.21.15.26:0/1468232881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9e0109110 0x7ff9e0053bf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8051fc0 tx=0x7ff9e8096010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c7c637582db97fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.126+0000 7f05faa00640 1 -- 172.21.15.26:0/2832832722 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4037050 con 0x7f05fc0530a0 2024-04-26T21:08:47.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.126+0000 7f0608a00640 1 -- 172.21.15.26:0/2832832722 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05fc04d5f0 con 0x7f05fc0530a0 2024-04-26T21:08:47.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.127+0000 7ff9e6000640 1 -- 172.21.15.26:0/1468232881 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e80a3070 con 0x7ff9e0109110 2024-04-26T21:08:47.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.126+0000 7f0608a00640 1 -- 172.21.15.26:0/2832832722 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f05fc04dbb0 con 0x7f05fc0530a0 2024-04-26T21:08:47.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.127+0000 7f05faa00640 1 -- 172.21.15.26:0/2832832722 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05f4043070 con 0x7f05fc0530a0 2024-04-26T21:08:47.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.127+0000 7f05faa00640 1 -- 172.21.15.26:0/2832832722 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f403cb10 con 0x7f05fc0530a0 2024-04-26T21:08:47.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.127+0000 7ff9e6000640 1 -- 172.21.15.26:0/1468232881 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9e8096e20 con 0x7ff9e0109110 2024-04-26T21:08:47.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.127+0000 7ff9e6000640 1 -- 172.21.15.26:0/1468232881 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e809e640 con 0x7ff9e0109110 2024-04-26T21:08:47.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.128+0000 7ff9ee600640 1 -- 172.21.15.26:0/1468232881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9e0109110 msgr2=0x7ff9e0053bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.128+0000 7ff9ee600640 1 --2- 172.21.15.26:0/1468232881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9e0109110 0x7ff9e0053bf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8051fc0 tx=0x7ff9e8096010 comp rx=0 tx=0).stop 2024-04-26T21:08:47.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.128+0000 7ff9ee600640 1 -- 172.21.15.26:0/1468232881 shutdown_connections 2024-04-26T21:08:47.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.128+0000 7ff9ee600640 1 --2- 172.21.15.26:0/1468232881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9e0109110 0x7ff9e0053bf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.128+0000 7ff9ee600640 1 -- 172.21.15.26:0/1468232881 >> 172.21.15.26:0/1468232881 conn(0x7ff9e0009fa0 msgr2=0x7ff9e000a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:47.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.128+0000 7ff9ee600640 1 -- 172.21.15.26:0/1468232881 shutdown_connections 2024-04-26T21:08:47.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.128+0000 7ff9ee600640 1 -- 172.21.15.26:0/1468232881 wait complete. 2024-04-26T21:08:47.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.128+0000 7f0608a00640 1 -- 172.21.15.26:0/2832832722 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05fc0379a0 con 0x7f05fc0530a0 2024-04-26T21:08:47.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.131+0000 7ff9ee600640 1 Processor -- start 2024-04-26T21:08:47.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.132+0000 7ff9ee600640 1 -- start start 2024-04-26T21:08:47.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.130+0000 7f05faa00640 1 -- 172.21.15.26:0/2832832722 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f05f403ccb0 con 0x7f05fc0530a0 2024-04-26T21:08:47.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.132+0000 7f05faa00640 1 --2- 172.21.15.26:0/2832832722 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05e406c8f0 0x7f05e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.133+0000 7f05faa00640 1 -- 172.21.15.26:0/2832832722 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f05f4078540 con 0x7f05fc0530a0 2024-04-26T21:08:47.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.133+0000 7f0602800640 1 --2- 172.21.15.26:0/2832832722 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05e406c8f0 0x7f05e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.134+0000 7ff9ee600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9e0105160 0x7ff9e00f6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.135+0000 7ff9ee600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9e0108ca0 0x7ff9e004e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.135+0000 7f0602800640 1 --2- 172.21.15.26:0/2832832722 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05e406c8f0 0x7f05e406edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0604000b60 tx=0x7f0604068050 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.135+0000 7ff9ee600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9e0109110 0x7ff9e004e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.135+0000 7ff9e7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9e0105160 0x7ff9e00f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.135+0000 7ff9e7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9e0105160 0x7ff9e00f6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.26:41940/0 (socket says 172.21.15.26:41940) 2024-04-26T21:08:47.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.136+0000 7ff9ee600640 1 -- 172.21.15.26:0/3278933449 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0052ce0 con 0x7ff9e0108ca0 2024-04-26T21:08:47.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.136+0000 7ff9ee600640 1 -- 172.21.15.26:0/3278933449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0053050 con 0x7ff9e0105160 2024-04-26T21:08:47.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.136+0000 7ff9ee600640 1 -- 172.21.15.26:0/3278933449 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e0006ee0 con 0x7ff9e0109110 2024-04-26T21:08:47.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.136+0000 7ff9e6a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9e0108ca0 0x7ff9e004e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.136+0000 7ff9e7400640 1 -- 172.21.15.26:0/3278933449 learned_addr learned my addr 172.21.15.26:0/3278933449 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:47.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.136+0000 7ff9e7e00640 1 --2- 172.21.15.26:0/3278933449 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9e0109110 0x7ff9e004e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.137+0000 7ff9e6a00640 1 -- 172.21.15.26:0/3278933449 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9e0109110 msgr2=0x7ff9e004e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.137+0000 7ff9e6a00640 1 --2- 172.21.15.26:0/3278933449 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9e0109110 0x7ff9e004e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.137+0000 7ff9e6a00640 1 -- 172.21.15.26:0/3278933449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9e0105160 msgr2=0x7ff9e00f6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.137+0000 7ff9e6a00640 1 --2- 172.21.15.26:0/3278933449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9e0105160 0x7ff9e00f6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.138+0000 7ff9e6a00640 1 -- 172.21.15.26:0/3278933449 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e00f6590 con 0x7ff9e0108ca0 2024-04-26T21:08:47.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.138+0000 7ff9e7400640 1 --2- 172.21.15.26:0/3278933449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9e0105160 0x7ff9e00f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:08:47.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.138+0000 7ff9e7e00640 1 --2- 172.21.15.26:0/3278933449 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9e0109110 0x7ff9e004e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:08:47.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.138+0000 7ff9e6a00640 1 --2- 172.21.15.26:0/3278933449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9e0108ca0 0x7ff9e004e000 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc00c370 tx=0x7ff9dc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.139+0000 7ff9db400640 1 -- 172.21.15.26:0/3278933449 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc017070 con 0x7ff9e0108ca0 2024-04-26T21:08:47.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.139+0000 7ff9db400640 1 -- 172.21.15.26:0/3278933449 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9dc0052f0 con 0x7ff9e0108ca0 2024-04-26T21:08:47.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.139+0000 7ff9db400640 1 -- 172.21.15.26:0/3278933449 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc003830 con 0x7ff9e0108ca0 2024-04-26T21:08:47.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.139+0000 7ff9ee600640 1 -- 172.21.15.26:0/3278933449 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e0035400 con 0x7ff9e0108ca0 2024-04-26T21:08:47.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.140+0000 7ff9ee600640 1 -- 172.21.15.26:0/3278933449 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e004bd20 con 0x7ff9e0108ca0 2024-04-26T21:08:47.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.142+0000 7ff9ee600640 1 -- 172.21.15.26:0/3278933449 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9e0035920 con 0x7ff9e0108ca0 2024-04-26T21:08:47.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.145+0000 7ff9db400640 1 -- 172.21.15.26:0/3278933449 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9dc01a070 con 0x7ff9e0108ca0 2024-04-26T21:08:47.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.146+0000 7ff9db400640 1 --2- 172.21.15.26:0/3278933449 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff9c806c8f0 0x7ff9c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.146+0000 7ff9db400640 1 -- 172.21.15.26:0/3278933449 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff9dc08ae60 con 0x7ff9e0108ca0 2024-04-26T21:08:47.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.147+0000 7ff9e7400640 1 --2- 172.21.15.26:0/3278933449 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff9c806c8f0 0x7ff9c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.149+0000 7ff9e7400640 1 --2- 172.21.15.26:0/3278933449 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff9c806c8f0 0x7ff9c806edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4007fd0 tx=0x7ff9d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.152+0000 7f05faa00640 1 -- 172.21.15.26:0/2832832722 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f05f407aba0 con 0x7f05fc0530a0 2024-04-26T21:08:47.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.154+0000 7ff9db400640 1 -- 172.21.15.26:0/3278933449 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff9e0035920 con 0x7ff9e0108ca0 2024-04-26T21:08:47.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.174+0000 7fa9b2e00640 1 Processor -- start 2024-04-26T21:08:47.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.174+0000 7fa9b2e00640 1 -- start start 2024-04-26T21:08:47.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.177+0000 7fa9b2e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9ac0b3df0 0x7fa9ac0d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.177+0000 7fa9b2e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa9ac167180 0x7fa9ac0d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.177+0000 7fa9b2e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9ac06e280 con 0x7fa9ac163730 2024-04-26T21:08:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.177+0000 7fa9b2e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fa9ac167680 con 0x7fa9ac167180 2024-04-26T21:08:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.177+0000 7fa9b2e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa9ac167960 con 0x7fa9ac0b3df0 2024-04-26T21:08:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.177+0000 7fa9abe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa9ac163730 0x7fa9ac0d0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44862/0 (socket says 172.21.15.26:44862) 2024-04-26T21:08:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.177+0000 7fa9abe00640 1 -- 172.21.15.26:0/3961033436 learned_addr learned my addr 172.21.15.26:0/3961033436 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.178+0000 7fa9ab400640 1 --2- 172.21.15.26:0/3961033436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9ac0b3df0 0x7fa9ac0d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.179+0000 7fa9b0e00640 1 --2- 172.21.15.26:0/3961033436 >> v2:172.21.15.26:3300/0 conn(0x7fa9ac167180 0x7fa9ac0d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.180+0000 7fa9b0e00640 1 -- 172.21.15.26:0/3961033436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9ac0b3df0 msgr2=0x7fa9ac0d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:47.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.180+0000 7fa9b0e00640 1 --2- 172.21.15.26:0/3961033436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9ac0b3df0 0x7fa9ac0d0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.180+0000 7fa9b0e00640 1 -- 172.21.15.26:0/3961033436 >> v1:172.21.15.26:6789/0 conn(0x7fa9ac163730 legacy=0x7fa9ac0d0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.180+0000 7fa9b0e00640 1 -- 172.21.15.26:0/3961033436 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ac167c40 con 0x7fa9ac167180 2024-04-26T21:08:47.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.181+0000 7f1d01800640 1 Processor -- start 2024-04-26T21:08:47.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.181+0000 7f1d01800640 1 -- start start 2024-04-26T21:08:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.182+0000 7fa9b0e00640 1 --2- 172.21.15.26:0/3961033436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9ac167180 0x7fa9ac0d1210 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa9980097a0 tx=0x7fa99802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef2e65b1e1a8c12 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.182+0000 7fa9aaa00640 1 -- 172.21.15.26:0/3961033436 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998037070 con 0x7fa9ac167180 2024-04-26T21:08:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.182+0000 7fa9aaa00640 1 -- 172.21.15.26:0/3961033436 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa998032440 con 0x7fa9ac167180 2024-04-26T21:08:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.182+0000 7fa9aaa00640 1 -- 172.21.15.26:0/3961033436 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998032810 con 0x7fa9ac167180 2024-04-26T21:08:47.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.183+0000 7fa9b2e00640 1 -- 172.21.15.26:0/3961033436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9ac167180 msgr2=0x7fa9ac0d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.183+0000 7fa9b2e00640 1 --2- 172.21.15.26:0/3961033436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9ac167180 0x7fa9ac0d1210 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa9980097a0 tx=0x7fa99802a180 comp rx=0 tx=0).stop 2024-04-26T21:08:47.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.184+0000 7fa9b2e00640 1 -- 172.21.15.26:0/3961033436 shutdown_connections 2024-04-26T21:08:47.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.184+0000 7f1d01800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cfc0b5ea0 0x7f1cfc073080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.184+0000 7fa9b2e00640 1 --2- 172.21.15.26:0/3961033436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9ac167180 0x7fa9ac0d1210 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.184+0000 7f1d01800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1cfc15c480 0x7f1cfc0735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.184+0000 7fa9b2e00640 1 --2- 172.21.15.26:0/3961033436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9ac0b3df0 0x7fa9ac0d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.184+0000 7fa9b2e00640 1 -- 172.21.15.26:0/3961033436 >> 172.21.15.26:0/3961033436 conn(0x7fa9ac0cd840 msgr2=0x7fa9ac0cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:47.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.185+0000 7fa9b2e00640 1 -- 172.21.15.26:0/3961033436 shutdown_connections 2024-04-26T21:08:47.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.185+0000 7f1cfa000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cfc0b5ea0 0x7f1cfc073080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.185+0000 7f1cfa000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cfc0b5ea0 0x7f1cfc073080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.26:46146/0 (socket says 172.21.15.26:46146) 2024-04-26T21:08:47.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.185+0000 7f1cfa000640 1 -- 172.21.15.26:0/1977965541 learned_addr learned my addr 172.21.15.26:0/1977965541 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:47.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.185+0000 7f1d01800640 1 -- 172.21.15.26:0/1977965541 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cfc070510 con 0x7f1cfc1522f0 2024-04-26T21:08:47.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.185+0000 7f1d01800640 1 -- 172.21.15.26:0/1977965541 --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f1cfc15bff0 con 0x7f1cfc15c480 2024-04-26T21:08:47.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.185+0000 7f1d01800640 1 -- 172.21.15.26:0/1977965541 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1cfc0702d0 con 0x7f1cfc0b5ea0 2024-04-26T21:08:47.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.185+0000 7fa9b2e00640 1 -- 172.21.15.26:0/3961033436 wait complete. 2024-04-26T21:08:47.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.186+0000 7f1cfb400640 1 --2- 172.21.15.26:0/1977965541 >> v2:172.21.15.26:3300/0 conn(0x7f1cfc15c480 0x7f1cfc0735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.186+0000 7f1cf9600640 1 -- 172.21.15.26:0/1977965541 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2293747237 0 0) 0x7f1cfc070510 con 0x7f1cfc1522f0 2024-04-26T21:08:47.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.186+0000 7f1cf9600640 1 -- 172.21.15.26:0/1977965541 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ce0003520 con 0x7f1cfc1522f0 2024-04-26T21:08:47.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.186+0000 7f1b64800640 1 Processor -- start 2024-04-26T21:08:47.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.188+0000 7fa9b2e00640 1 Processor -- start 2024-04-26T21:08:47.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.187+0000 7f1cfa000640 1 -- 172.21.15.26:0/1977965541 >> v2:172.21.15.26:3300/0 conn(0x7f1cfc15c480 msgr2=0x7f1cfc0735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:47.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.187+0000 7f1cfa000640 1 --2- 172.21.15.26:0/1977965541 >> v2:172.21.15.26:3300/0 conn(0x7f1cfc15c480 0x7f1cfc0735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.187+0000 7f1cfa000640 1 -- 172.21.15.26:0/1977965541 >> v1:172.21.15.26:6789/0 conn(0x7f1cfc1522f0 legacy=0x7f1cfc06fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.187+0000 7f1cfa000640 1 -- 172.21.15.26:0/1977965541 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cfc15b9a0 con 0x7f1cfc0b5ea0 2024-04-26T21:08:47.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.188+0000 7f1cfb400640 1 --2- 172.21.15.26:0/1977965541 >> v2:172.21.15.26:3300/0 conn(0x7f1cfc15c480 0x7f1cfc0735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:47.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.188+0000 7fa9b2e00640 1 -- start start 2024-04-26T21:08:47.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.188+0000 7f1cfa000640 1 --2- 172.21.15.26:0/1977965541 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cfc0b5ea0 0x7f1cfc073080 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009870 tx=0x7f1cec02a150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7516a71b701120cc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.187+0000 7f1b64800640 1 -- start start 2024-04-26T21:08:47.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.188+0000 7f1b64800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1b5c0d39f0 0x7f1b5c0d5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.189+0000 7f1b64800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b5c15f710 0x7f1b5c0d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.189+0000 7f1b64800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b5c0d0ef0 con 0x7f1b5c0d3dd0 2024-04-26T21:08:47.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.189+0000 7f1b64800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f1b5c0b4900 con 0x7f1b5c0d39f0 2024-04-26T21:08:47.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.189+0000 7f1b64800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5c0b4be0 con 0x7f1b5c15f710 2024-04-26T21:08:47.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.189+0000 7f1b61e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1b5c0d39f0 0x7f1b5c0d5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.189+0000 7f1b61e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1b5c0d39f0 0x7f1b5c0d5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34820/0 (socket says 172.21.15.26:34820) 2024-04-26T21:08:47.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.189+0000 7f1b61e00640 1 -- 172.21.15.26:0/4029606035 learned_addr learned my addr 172.21.15.26:0/4029606035 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:47.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.189+0000 7f1b62800640 1 --2- 172.21.15.26:0/4029606035 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b5c15f710 0x7f1b5c0d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.189+0000 7f1cf9600640 1 -- 172.21.15.26:0/1977965541 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec037070 con 0x7f1cfc0b5ea0 2024-04-26T21:08:47.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.189+0000 7f1cf9600640 1 -- 172.21.15.26:0/1977965541 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cec032440 con 0x7f1cfc0b5ea0 2024-04-26T21:08:47.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.189+0000 7f1cf9600640 1 -- 172.21.15.26:0/1977965541 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec032850 con 0x7f1cfc0b5ea0 2024-04-26T21:08:47.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.190+0000 7fa9b2e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9ac0b3df0 0x7fa9ac152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.190+0000 7f1b61e00640 1 -- 172.21.15.26:0/4029606035 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b5c15f710 msgr2=0x7f1b5c0d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:47.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.190+0000 7f1b61e00640 1 --2- 172.21.15.26:0/4029606035 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b5c15f710 0x7f1b5c0d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.190+0000 7f1b61e00640 1 -- 172.21.15.26:0/4029606035 >> v1:172.21.15.86:6789/0 conn(0x7f1b5c0d3dd0 legacy=0x7f1b5c15f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.190+0000 7f1b61e00640 1 -- 172.21.15.26:0/4029606035 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5c0b4ec0 con 0x7f1b5c0d39f0 2024-04-26T21:08:47.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.190+0000 7f1d01800640 1 -- 172.21.15.26:0/1977965541 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cfc0b5ea0 msgr2=0x7f1cfc073080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.190+0000 7f1d01800640 1 --2- 172.21.15.26:0/1977965541 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cfc0b5ea0 0x7f1cfc073080 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009870 tx=0x7f1cec02a150 comp rx=0 tx=0).stop 2024-04-26T21:08:47.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.191+0000 7f1d01800640 1 -- 172.21.15.26:0/1977965541 shutdown_connections 2024-04-26T21:08:47.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.191+0000 7f1d01800640 1 --2- 172.21.15.26:0/1977965541 >> v2:172.21.15.26:3300/0 conn(0x7f1cfc15c480 0x7f1cfc0735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.191+0000 7f1d01800640 1 --2- 172.21.15.26:0/1977965541 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cfc0b5ea0 0x7f1cfc073080 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.191+0000 7f1d01800640 1 -- 172.21.15.26:0/1977965541 >> 172.21.15.26:0/1977965541 conn(0x7f1cfc06ca30 msgr2=0x7f1cfc0b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:47.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.191+0000 7fa9b2e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9ac163730 0x7fa9ac1f9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.191+0000 7fa9b2e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9ac167180 0x7fa9ac164290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.191+0000 7fa9b2e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac06e8e0 con 0x7fa9ac0b3df0 2024-04-26T21:08:47.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.191+0000 7fa9b2e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac06e760 con 0x7fa9ac163730 2024-04-26T21:08:47.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.191+0000 7fa9b2e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa9ac06ea60 con 0x7fa9ac167180 2024-04-26T21:08:47.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.191+0000 7f1d01800640 1 -- 172.21.15.26:0/1977965541 shutdown_connections 2024-04-26T21:08:47.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.191+0000 7fa9b0e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9ac167180 0x7fa9ac164290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.191+0000 7fa9abe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9ac0b3df0 0x7fa9ac152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.191+0000 7fa9ab400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9ac163730 0x7fa9ac1f9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.191+0000 7fa9b0e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9ac167180 0x7fa9ac164290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.26:46174/0 (socket says 172.21.15.26:46174) 2024-04-26T21:08:47.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.191+0000 7fa9b0e00640 1 -- 172.21.15.26:0/2839877441 learned_addr learned my addr 172.21.15.26:0/2839877441 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:47.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.192+0000 7f1d01800640 1 -- 172.21.15.26:0/1977965541 wait complete. 2024-04-26T21:08:47.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.192+0000 7f1b61e00640 1 --2- 172.21.15.26:0/4029606035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b5c0d39f0 0x7f1b5c0d5cc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009e50 tx=0x7f1b5802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af52961770b02a52 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.192+0000 7fa9abe00640 1 -- 172.21.15.26:0/2839877441 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9ac167180 msgr2=0x7fa9ac164290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.192+0000 7fa9abe00640 1 --2- 172.21.15.26:0/2839877441 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9ac167180 0x7fa9ac164290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.192+0000 7fa9abe00640 1 -- 172.21.15.26:0/2839877441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9ac163730 msgr2=0x7fa9ac1f9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.192+0000 7fa9abe00640 1 --2- 172.21.15.26:0/2839877441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9ac163730 0x7fa9ac1f9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.192+0000 7fa9abe00640 1 -- 172.21.15.26:0/2839877441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ac164eb0 con 0x7fa9ac0b3df0 2024-04-26T21:08:47.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.193+0000 7f1b60a00640 1 -- 172.21.15.26:0/4029606035 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b58036070 con 0x7f1b5c0d39f0 2024-04-26T21:08:47.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.193+0000 7f1b60a00640 1 -- 172.21.15.26:0/4029606035 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b58031600 con 0x7f1b5c0d39f0 2024-04-26T21:08:47.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.193+0000 7f1b60a00640 1 -- 172.21.15.26:0/4029606035 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b58036070 con 0x7f1b5c0d39f0 2024-04-26T21:08:47.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.193+0000 7fa9abe00640 1 --2- 172.21.15.26:0/2839877441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9ac0b3df0 0x7fa9ac152f50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400ca50 tx=0x7fa9a400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.194+0000 7fa9a3e00640 1 -- 172.21.15.26:0/2839877441 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4017070 con 0x7fa9ac0b3df0 2024-04-26T21:08:47.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.194+0000 7fa9a3e00640 1 -- 172.21.15.26:0/2839877441 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a400abb0 con 0x7fa9ac0b3df0 2024-04-26T21:08:47.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.194+0000 7fa9a3e00640 1 -- 172.21.15.26:0/2839877441 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4012400 con 0x7fa9ac0b3df0 2024-04-26T21:08:47.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.195+0000 7f1b64800640 1 -- 172.21.15.26:0/4029606035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b5c0d39f0 msgr2=0x7f1b5c0d5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.195+0000 7f1b64800640 1 --2- 172.21.15.26:0/4029606035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b5c0d39f0 0x7f1b5c0d5cc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009e50 tx=0x7f1b5802a710 comp rx=0 tx=0).stop 2024-04-26T21:08:47.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.195+0000 7fa9b2e00640 1 -- 172.21.15.26:0/2839877441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9ac165190 con 0x7fa9ac0b3df0 2024-04-26T21:08:47.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.195+0000 7fa9b2e00640 1 -- 172.21.15.26:0/2839877441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9ac165750 con 0x7fa9ac0b3df0 2024-04-26T21:08:47.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.195+0000 7f1d01800640 1 Processor -- start 2024-04-26T21:08:47.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.195+0000 7f1d01800640 1 -- start start 2024-04-26T21:08:47.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.196+0000 7f1b64800640 1 -- 172.21.15.26:0/4029606035 shutdown_connections 2024-04-26T21:08:47.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.196+0000 7f1b64800640 1 --2- 172.21.15.26:0/4029606035 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b5c15f710 0x7f1b5c0d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.196+0000 7f1b64800640 1 --2- 172.21.15.26:0/4029606035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b5c0d39f0 0x7f1b5c0d5cc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.196+0000 7f1b64800640 1 -- 172.21.15.26:0/4029606035 >> 172.21.15.26:0/4029606035 conn(0x7f1b5c0d51a0 msgr2=0x7f1b5c0d55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:47.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.196+0000 7f1b64800640 1 -- 172.21.15.26:0/4029606035 shutdown_connections 2024-04-26T21:08:47.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.197+0000 7f1d01800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cfc0b5ea0 0x7f1cfc1e3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.197+0000 7f1d01800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cfc1522f0 0x7f1cfc15a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.196+0000 7f1b64800640 1 -- 172.21.15.26:0/4029606035 wait complete. 2024-04-26T21:08:47.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.197+0000 7f1d01800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc15c480 0x7f1cfc205350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.197+0000 7f1d01800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1cfc1526d0 con 0x7f1cfc15c480 2024-04-26T21:08:47.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.197+0000 7f1d01800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1cfc0707f0 con 0x7f1cfc0b5ea0 2024-04-26T21:08:47.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.197+0000 7f1d01800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1cfc0b3200 con 0x7f1cfc1522f0 2024-04-26T21:08:47.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.197+0000 7f1cfb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc15c480 0x7f1cfc205350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.197+0000 7f1cfb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc15c480 0x7f1cfc205350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34850/0 (socket says 172.21.15.26:34850) 2024-04-26T21:08:47.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.197+0000 7f1cfb400640 1 -- 172.21.15.26:0/2956515949 learned_addr learned my addr 172.21.15.26:0/2956515949 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:47.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.198+0000 7f1cfa000640 1 --2- 172.21.15.26:0/2956515949 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cfc1522f0 0x7f1cfc15a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.198+0000 7f1cfaa00640 1 --2- 172.21.15.26:0/2956515949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cfc0b5ea0 0x7f1cfc1e3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.198+0000 7f1cfa000640 1 -- 172.21.15.26:0/2956515949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cfc0b5ea0 msgr2=0x7f1cfc1e3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.198+0000 7f1cfa000640 1 --2- 172.21.15.26:0/2956515949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cfc0b5ea0 0x7f1cfc1e3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.198+0000 7f1cfa000640 1 -- 172.21.15.26:0/2956515949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc15c480 msgr2=0x7f1cfc205350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.198+0000 7f1cfa000640 1 --2- 172.21.15.26:0/2956515949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc15c480 0x7f1cfc205350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.198+0000 7f1cfa000640 1 -- 172.21.15.26:0/2956515949 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cfc1e4510 con 0x7f1cfc1522f0 2024-04-26T21:08:47.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.197+0000 7fa9b2e00640 1 -- 172.21.15.26:0/2839877441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9ac0b6b50 con 0x7fa9ac0b3df0 2024-04-26T21:08:47.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.200+0000 7f1cfa000640 1 --2- 172.21.15.26:0/2956515949 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cfc1522f0 0x7f1cfc15a510 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0b0a00 tx=0x7f1cec0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.200+0000 7f1cf2a00640 1 -- 172.21.15.26:0/2956515949 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec041020 con 0x7f1cfc1522f0 2024-04-26T21:08:47.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.201+0000 7f1d01800640 1 -- 172.21.15.26:0/2956515949 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cfc1e47f0 con 0x7f1cfc1522f0 2024-04-26T21:08:47.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.201+0000 7f1b64800640 1 Processor -- start 2024-04-26T21:08:47.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.201+0000 7f1b64800640 1 -- start start 2024-04-26T21:08:47.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.201+0000 7f1cf2a00640 1 -- 172.21.15.26:0/2956515949 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cec03d620 con 0x7f1cfc1522f0 2024-04-26T21:08:47.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.201+0000 7f1d01800640 1 -- 172.21.15.26:0/2956515949 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cfc1e4db0 con 0x7f1cfc1522f0 2024-04-26T21:08:47.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.203+0000 7f1b64800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b5c0d39f0 0x7f1b5c15a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.204+0000 7f1b61e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b5c0d39f0 0x7f1b5c15a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.204+0000 7f1b61e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b5c0d39f0 0x7f1b5c15a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34866/0 (socket says 172.21.15.26:34866) 2024-04-26T21:08:47.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.204+0000 7f1b61e00640 1 -- 172.21.15.26:0/275757033 learned_addr learned my addr 172.21.15.26:0/275757033 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:47.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.205+0000 7f1b64800640 1 --2- 172.21.15.26:0/275757033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b5c0d3dd0 0x7f1b5c0ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.205+0000 7f1b64800640 1 --2- 172.21.15.26:0/275757033 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b5c15f710 0x7f1b5c0b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.203+0000 7f1cf2a00640 1 -- 172.21.15.26:0/2956515949 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec041020 con 0x7f1cfc1522f0 2024-04-26T21:08:47.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.205+0000 7f1b64800640 1 -- 172.21.15.26:0/275757033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c0b57b0 con 0x7f1b5c0d39f0 2024-04-26T21:08:47.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.205+0000 7f1b61400640 1 --2- 172.21.15.26:0/275757033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b5c0d3dd0 0x7f1b5c0ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.206+0000 7f1b62800640 1 --2- 172.21.15.26:0/275757033 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b5c15f710 0x7f1b5c0b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.206+0000 7f1b64800640 1 -- 172.21.15.26:0/275757033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c0b5630 con 0x7f1b5c0d3dd0 2024-04-26T21:08:47.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.206+0000 7f1b64800640 1 -- 172.21.15.26:0/275757033 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5c0b5930 con 0x7f1b5c15f710 2024-04-26T21:08:47.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.205+0000 7f1cf2a00640 1 -- 172.21.15.26:0/2956515949 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1cec044020 con 0x7f1cfc1522f0 2024-04-26T21:08:47.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.206+0000 7f1ce7e00640 1 -- 172.21.15.26:0/2956515949 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd4005290 con 0x7f1cfc1522f0 2024-04-26T21:08:47.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.207+0000 7f1cf2a00640 1 --2- 172.21.15.26:0/2956515949 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1cdc06caf0 0x7f1cdc06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.207+0000 7f1cf2a00640 1 -- 172.21.15.26:0/2956515949 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1cec077c00 con 0x7f1cfc1522f0 2024-04-26T21:08:47.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.208+0000 7f1b61e00640 1 -- 172.21.15.26:0/275757033 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b5c15f710 msgr2=0x7f1b5c0b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.208+0000 7f1b61e00640 1 --2- 172.21.15.26:0/275757033 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b5c15f710 0x7f1b5c0b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.208+0000 7f1b61e00640 1 -- 172.21.15.26:0/275757033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b5c0d3dd0 msgr2=0x7f1b5c0ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.209+0000 7f1b61e00640 1 --2- 172.21.15.26:0/275757033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b5c0d3dd0 0x7f1b5c0ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.208+0000 7fa9a3e00640 1 -- 172.21.15.26:0/2839877441 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9a401a070 con 0x7fa9ac0b3df0 2024-04-26T21:08:47.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.209+0000 7f1b61e00640 1 -- 172.21.15.26:0/275757033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5c160770 con 0x7f1b5c0d39f0 2024-04-26T21:08:47.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.209+0000 7f1cfaa00640 1 --2- 172.21.15.26:0/2956515949 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1cdc06caf0 0x7f1cdc06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.209+0000 7f1b62800640 1 --2- 172.21.15.26:0/275757033 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b5c15f710 0x7f1b5c0b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:08:47.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.209+0000 7f1b61400640 1 --2- 172.21.15.26:0/275757033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b5c0d3dd0 0x7f1b5c0ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:08:47.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.210+0000 7f1b61e00640 1 --2- 172.21.15.26:0/275757033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b5c0d39f0 0x7f1b5c15a820 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1b5802ad00 tx=0x7f1b58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.210+0000 7fa9a3e00640 1 --2- 172.21.15.26:0/2839877441 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa98c06c970 0x7fa98c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.211+0000 7f1b56000640 1 -- 172.21.15.26:0/275757033 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5803f020 con 0x7f1b5c0d39f0 2024-04-26T21:08:47.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.211+0000 7f1b56000640 1 -- 172.21.15.26:0/275757033 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b58037040 con 0x7f1b5c0d39f0 2024-04-26T21:08:47.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.211+0000 7f1b56000640 1 -- 172.21.15.26:0/275757033 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b58036070 con 0x7f1b5c0d39f0 2024-04-26T21:08:47.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.211+0000 7f1b64800640 1 -- 172.21.15.26:0/275757033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5c160a50 con 0x7f1b5c0d39f0 2024-04-26T21:08:47.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.211+0000 7f1b64800640 1 -- 172.21.15.26:0/275757033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b5c161010 con 0x7f1b5c0d39f0 2024-04-26T21:08:47.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.211+0000 7fa9a3e00640 1 -- 172.21.15.26:0/2839877441 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa9a408b7d0 con 0x7fa9ac0b3df0 2024-04-26T21:08:47.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.211+0000 7fa9a3e00640 1 -- 172.21.15.26:0/2839877441 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa9ac0b6b50 con 0x7fa9ac0b3df0 2024-04-26T21:08:47.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.212+0000 7f1cfaa00640 1 --2- 172.21.15.26:0/2956515949 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1cdc06caf0 0x7f1cdc06efb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4001d10 tx=0x7f1cf4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.215+0000 7fa9ab400640 1 --2- 172.21.15.26:0/2839877441 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa98c06c970 0x7fa98c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.215+0000 7f1b56000640 1 -- 172.21.15.26:0/275757033 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b58042020 con 0x7f1b5c0d39f0 2024-04-26T21:08:47.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.216+0000 7fa9ab400640 1 --2- 172.21.15.26:0/2839877441 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa98c06c970 0x7fa98c06ee30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0d0280 tx=0x7fa99c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.216+0000 7f1b56000640 1 --2- 172.21.15.26:0/275757033 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1b4006c940 0x7f1b4006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.216+0000 7f1b64800640 1 -- 172.21.15.26:0/275757033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b5c075e60 con 0x7f1b5c0d39f0 2024-04-26T21:08:47.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.217+0000 7f1b56000640 1 -- 172.21.15.26:0/275757033 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1b580ac260 con 0x7f1b5c0d39f0 2024-04-26T21:08:47.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.217+0000 7f1b61400640 1 --2- 172.21.15.26:0/275757033 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1b4006c940 0x7f1b4006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.220+0000 7f1b61400640 1 --2- 172.21.15.26:0/275757033 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1b4006c940 0x7f1b4006ee00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c162bf0 tx=0x7f1b4c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.223+0000 7f1cf2a00640 1 -- 172.21.15.26:0/2956515949 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1cec04b030 con 0x7f1cfc1522f0 2024-04-26T21:08:47.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.224+0000 7f1b56000640 1 -- 172.21.15.26:0/275757033 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1b5c075e60 con 0x7f1b5c0d39f0 2024-04-26T21:08:47.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.254+0000 7f066a000640 1 Processor -- start 2024-04-26T21:08:47.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.254+0000 7f066a000640 1 -- start start 2024-04-26T21:08:47.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.256+0000 7f066a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f066415f680 0x7f06640d5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.256+0000 7f066a000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f06640d1b80 0x7f06640d1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.256+0000 7f066a000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06640d6110 con 0x7f0664160500 2024-04-26T21:08:47.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.256+0000 7f066a000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f06640d2510 con 0x7f06640d1b80 2024-04-26T21:08:47.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.256+0000 7f066a000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f06640d0870 con 0x7f066415f680 2024-04-26T21:08:47.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.257+0000 7f0663e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f06640d1b80 0x7f06640d1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.257+0000 7f0663e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f06640d1b80 0x7f06640d1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34868/0 (socket says 172.21.15.26:34868) 2024-04-26T21:08:47.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.257+0000 7f0663e00640 1 -- 172.21.15.26:0/65369225 learned_addr learned my addr 172.21.15.26:0/65369225 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:47.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.257+0000 7f0663400640 1 --2- 172.21.15.26:0/65369225 >> v2:172.21.15.86:3300/0 conn(0x7f066415f680 0x7f06640d5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.258+0000 7f0663e00640 1 -- 172.21.15.26:0/65369225 >> v2:172.21.15.86:3300/0 conn(0x7f066415f680 msgr2=0x7f06640d5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:47.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.258+0000 7f0663e00640 1 --2- 172.21.15.26:0/65369225 >> v2:172.21.15.86:3300/0 conn(0x7f066415f680 0x7f06640d5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.258+0000 7f0663e00640 1 -- 172.21.15.26:0/65369225 >> v1:172.21.15.86:6789/0 conn(0x7f0664160500 legacy=0x7f06640d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:47.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.258+0000 7f0663e00640 1 -- 172.21.15.26:0/65369225 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06640cff40 con 0x7f06640d1b80 2024-04-26T21:08:47.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.259+0000 7f0663e00640 1 --2- 172.21.15.26:0/65369225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06640d1b80 0x7f06640d1fd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f06640b7d20 tx=0x7f065c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3daf4515050479a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.260+0000 7f0662a00640 1 -- 172.21.15.26:0/65369225 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c037070 con 0x7f06640d1b80 2024-04-26T21:08:47.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.260+0000 7f0662a00640 1 -- 172.21.15.26:0/65369225 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f065c032440 con 0x7f06640d1b80 2024-04-26T21:08:47.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.260+0000 7f0662a00640 1 -- 172.21.15.26:0/65369225 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c032810 con 0x7f06640d1b80 2024-04-26T21:08:47.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.261+0000 7f066a000640 1 -- 172.21.15.26:0/65369225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06640d1b80 msgr2=0x7f06640d1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.261+0000 7f066a000640 1 --2- 172.21.15.26:0/65369225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06640d1b80 0x7f06640d1fd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f06640b7d20 tx=0x7f065c02a230 comp rx=0 tx=0).stop 2024-04-26T21:08:47.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.262+0000 7f066a000640 1 -- 172.21.15.26:0/65369225 shutdown_connections 2024-04-26T21:08:47.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.262+0000 7f066a000640 1 --2- 172.21.15.26:0/65369225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06640d1b80 0x7f06640d1fd0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.262+0000 7f066a000640 1 --2- 172.21.15.26:0/65369225 >> v2:172.21.15.86:3300/0 conn(0x7f066415f680 0x7f06640d5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.262+0000 7f066a000640 1 -- 172.21.15.26:0/65369225 >> 172.21.15.26:0/65369225 conn(0x7f066409e490 msgr2=0x7f06640d51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:47.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.262+0000 7f066a000640 1 -- 172.21.15.26:0/65369225 shutdown_connections 2024-04-26T21:08:47.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.262+0000 7f066a000640 1 -- 172.21.15.26:0/65369225 wait complete. 2024-04-26T21:08:47.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.265+0000 7f066a000640 1 Processor -- start 2024-04-26T21:08:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.265+0000 7fee99e00640 1 Processor -- start 2024-04-26T21:08:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.265+0000 7fee99e00640 1 -- start start 2024-04-26T21:08:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.265+0000 7f066a000640 1 -- start start 2024-04-26T21:08:47.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.267+0000 7f066a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06640d1b80 0x7f066415c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.267+0000 7f066a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f066415f680 0x7f066415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.268+0000 7f066a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0664160500 0x7f066415dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.267+0000 7fee99e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee940cf6f0 0x7fee94145a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.267+0000 7fee99e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fee94071150 0x7fee940d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.268+0000 7fee93400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fee94071150 0x7fee940d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.268+0000 7f066a000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0664160940 con 0x7f0664160500 2024-04-26T21:08:47.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.268+0000 7f066a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f06640d0500 con 0x7f06640d1b80 2024-04-26T21:08:47.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.268+0000 7f066a000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f06640d2d80 con 0x7f066415f680 2024-04-26T21:08:47.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.268+0000 7f0663400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06640d1b80 0x7f066415c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.268+0000 7f0663400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06640d1b80 0x7f066415c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.26:41982/0 (socket says 172.21.15.26:41982) 2024-04-26T21:08:47.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.268+0000 7f0663400640 1 -- 172.21.15.26:0/1894633067 learned_addr learned my addr 172.21.15.26:0/1894633067 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:47.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.268+0000 7f065aa00640 1 --2- 172.21.15.26:0/1894633067 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f066415f680 0x7f066415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.268+0000 7f0663e00640 1 --2- 172.21.15.26:0/1894633067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0664160500 0x7f066415dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.268+0000 7fee93400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fee94071150 0x7fee940d0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34888/0 (socket says 172.21.15.26:34888) 2024-04-26T21:08:47.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.268+0000 7fee99e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee940cfbb0 con 0x7fee94070280 2024-04-26T21:08:47.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.268+0000 7fee99e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fee940cfe90 con 0x7fee94071150 2024-04-26T21:08:47.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.268+0000 7fee99e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fee940d0170 con 0x7fee940cf6f0 2024-04-26T21:08:47.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.268+0000 7fee93400640 1 -- 172.21.15.26:0/3032578122 learned_addr learned my addr 172.21.15.26:0/3032578122 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:47.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.269+0000 7f0663400640 1 -- 172.21.15.26:0/1894633067 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f066415f680 msgr2=0x7f066415a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.269+0000 7f0663400640 1 --2- 172.21.15.26:0/1894633067 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f066415f680 0x7f066415a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.269+0000 7f0663400640 1 -- 172.21.15.26:0/1894633067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0664160500 msgr2=0x7f066415dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.269+0000 7fee92a00640 1 --2- 172.21.15.26:0/3032578122 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee940cf6f0 0x7fee94145a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.269+0000 7fee93400640 1 -- 172.21.15.26:0/3032578122 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee940cf6f0 msgr2=0x7fee94145a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:47.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.269+0000 7fee93400640 1 --2- 172.21.15.26:0/3032578122 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee940cf6f0 0x7fee94145a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.269+0000 7fee93400640 1 -- 172.21.15.26:0/3032578122 >> v1:172.21.15.86:6789/0 conn(0x7fee94070280 legacy=0x7fee94070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:47.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.269+0000 7fee93400640 1 -- 172.21.15.26:0/3032578122 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee940d0450 con 0x7fee94071150 2024-04-26T21:08:47.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.270+0000 7f0663400640 1 --2- 172.21.15.26:0/1894633067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0664160500 0x7f066415dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.270+0000 7f0663400640 1 -- 172.21.15.26:0/1894633067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066415cc20 con 0x7f06640d1b80 2024-04-26T21:08:47.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.270+0000 7f0663e00640 1 --2- 172.21.15.26:0/1894633067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0664160500 0x7f066415dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:08:47.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.271+0000 7f0663400640 1 --2- 172.21.15.26:0/1894633067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06640d1b80 0x7f066415c320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f064c00d830 tx=0x7f064c00dcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.271+0000 7fee93400640 1 --2- 172.21.15.26:0/3032578122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee94071150 0x7fee940d0c50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fee8c009e50 tx=0x7fee8c02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5a4920798525ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.272+0000 7f69c2000640 1 Processor -- start 2024-04-26T21:08:47.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.272+0000 7fee91600640 1 -- 172.21.15.26:0/3032578122 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c036070 con 0x7fee94071150 2024-04-26T21:08:47.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.272+0000 7f065be00640 1 -- 172.21.15.26:0/1894633067 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c00b6c0 con 0x7f06640d1b80 2024-04-26T21:08:47.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.272+0000 7f065be00640 1 -- 172.21.15.26:0/1894633067 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f064c019070 con 0x7f06640d1b80 2024-04-26T21:08:47.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.272+0000 7fee91600640 1 -- 172.21.15.26:0/3032578122 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee8c031600 con 0x7fee94071150 2024-04-26T21:08:47.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.273+0000 7f066a000640 1 -- 172.21.15.26:0/1894633067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066415cf60 con 0x7f06640d1b80 2024-04-26T21:08:47.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.273+0000 7f066a000640 1 -- 172.21.15.26:0/1894633067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f066415d7a0 con 0x7f06640d1b80 2024-04-26T21:08:47.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.273+0000 7fee99e00640 1 -- 172.21.15.26:0/3032578122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee94071150 msgr2=0x7fee940d0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.273+0000 7fee99e00640 1 --2- 172.21.15.26:0/3032578122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee94071150 0x7fee940d0c50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fee8c009e50 tx=0x7fee8c02a6d0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.273+0000 7f065be00640 1 -- 172.21.15.26:0/1894633067 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c00ba60 con 0x7f06640d1b80 2024-04-26T21:08:47.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.273+0000 7fee99e00640 1 -- 172.21.15.26:0/3032578122 shutdown_connections 2024-04-26T21:08:47.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.273+0000 7fee99e00640 1 --2- 172.21.15.26:0/3032578122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee94071150 0x7fee940d0c50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.273+0000 7fee99e00640 1 --2- 172.21.15.26:0/3032578122 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee940cf6f0 0x7fee94145a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.273+0000 7fee99e00640 1 -- 172.21.15.26:0/3032578122 >> 172.21.15.26:0/3032578122 conn(0x7fee940d1e10 msgr2=0x7fee940d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:47.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.272+0000 7f69c2000640 1 -- start start 2024-04-26T21:08:47.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.274+0000 7fee99e00640 1 -- 172.21.15.26:0/3032578122 shutdown_connections 2024-04-26T21:08:47.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.274+0000 7fee99e00640 1 -- 172.21.15.26:0/3032578122 wait complete. 2024-04-26T21:08:47.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.275+0000 7f69c2000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f69bc163730 0x7f69bc0d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.275+0000 7f69c2000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69bc0b3df0 0x7f69bc0d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.275+0000 7f69c2000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69bc167180 0x7f69bc0d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.275+0000 7f69c2000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f69bc06e280 con 0x7f69bc0b3df0 2024-04-26T21:08:47.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.275+0000 7f69c2000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f69bc167680 con 0x7f69bc163730 2024-04-26T21:08:47.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.275+0000 7f69c2000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f69bc167960 con 0x7f69bc167180 2024-04-26T21:08:47.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.276+0000 7f69baa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69bc0b3df0 0x7f69bc0d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.276+0000 7f69baa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69bc0b3df0 0x7f69bc0d0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34904/0 (socket says 172.21.15.26:34904) 2024-04-26T21:08:47.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.276+0000 7f69baa00640 1 -- 172.21.15.26:0/927281906 learned_addr learned my addr 172.21.15.26:0/927281906 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:47.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.276+0000 7f69bb400640 1 --2- 172.21.15.26:0/927281906 >> v2:172.21.15.86:3300/0 conn(0x7f69bc163730 0x7f69bc0d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.276+0000 7f69bbe00640 1 --2- 172.21.15.26:0/927281906 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69bc167180 0x7f69bc0d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.276+0000 7f066a000640 1 -- 172.21.15.26:0/1894633067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f066406a700 con 0x7f06640d1b80 2024-04-26T21:08:47.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.277+0000 7f69bb400640 1 -- 172.21.15.26:0/927281906 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69bc167180 msgr2=0x7f69bc0d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:47.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.277+0000 7f69bb400640 1 --2- 172.21.15.26:0/927281906 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69bc167180 0x7f69bc0d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.277+0000 7f69bb400640 1 -- 172.21.15.26:0/927281906 >> v2:172.21.15.26:3300/0 conn(0x7f69bc0b3df0 msgr2=0x7f69bc0d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:47.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.277+0000 7f69bb400640 1 --2- 172.21.15.26:0/927281906 >> v2:172.21.15.26:3300/0 conn(0x7f69bc0b3df0 0x7f69bc0d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.277+0000 7f69bb400640 1 -- 172.21.15.26:0/927281906 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69bc167c40 con 0x7f69bc163730 2024-04-26T21:08:47.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.277+0000 7f69baa00640 1 --2- 172.21.15.26:0/927281906 >> v2:172.21.15.26:3300/0 conn(0x7f69bc0b3df0 0x7f69bc0d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:08:47.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.277+0000 7f065be00640 1 -- 172.21.15.26:0/1894633067 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f064c00bc00 con 0x7f06640d1b80 2024-04-26T21:08:47.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.279+0000 7fee99e00640 1 Processor -- start 2024-04-26T21:08:47.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.279+0000 7f065be00640 1 --2- 172.21.15.26:0/1894633067 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f064006c940 0x7f064006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.279+0000 7f065be00640 1 -- 172.21.15.26:0/1894633067 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f064c057d20 con 0x7f06640d1b80 2024-04-26T21:08:47.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.279+0000 7f065aa00640 1 --2- 172.21.15.26:0/1894633067 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f064006c940 0x7f064006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.280+0000 7fee99e00640 1 -- start start 2024-04-26T21:08:47.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.280+0000 7f69bb400640 1 --2- 172.21.15.26:0/927281906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69bc163730 0x7f69bc0d0410 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0d1920 tx=0x7f69b402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d1d045f43a0819f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.281+0000 7f065aa00640 1 --2- 172.21.15.26:0/1894633067 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f064006c940 0x7f064006ee00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f06640cc080 tx=0x7f065400d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.281+0000 7fee99e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee94070280 0x7fee941415a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.281+0000 7fee99e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee94071150 0x7fee9415b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.281+0000 7fee99e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee940cf6f0 0x7fee94140f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.281+0000 7fee99e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fee94070860 con 0x7fee94070280 2024-04-26T21:08:47.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.281+0000 7fee99e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee940706e0 con 0x7fee940cf6f0 2024-04-26T21:08:47.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.281+0000 7fee99e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fee940b4e00 con 0x7fee94071150 2024-04-26T21:08:47.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.281+0000 7fee92a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee94070280 0x7fee941415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.281+0000 7fee92a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee94070280 0x7fee941415a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34918/0 (socket says 172.21.15.26:34918) 2024-04-26T21:08:47.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.281+0000 7fee92a00640 1 -- 172.21.15.26:0/1334315238 learned_addr learned my addr 172.21.15.26:0/1334315238 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.282+0000 7fee93400640 1 --2- 172.21.15.26:0/1334315238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee940cf6f0 0x7fee94140f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.282+0000 7fee92a00640 1 -- 172.21.15.26:0/1334315238 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee94071150 msgr2=0x7fee9415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.282+0000 7fee92a00640 1 --2- 172.21.15.26:0/1334315238 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee94071150 0x7fee9415b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.282+0000 7fee92a00640 1 -- 172.21.15.26:0/1334315238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee940cf6f0 msgr2=0x7fee94140f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.282+0000 7fee92a00640 1 --2- 172.21.15.26:0/1334315238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee940cf6f0 0x7fee94140f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.282+0000 7fee92a00640 1 -- 172.21.15.26:0/1334315238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee94141b80 con 0x7fee94070280 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.283+0000 7fee92a00640 1 --2- 172.21.15.26:0/1334315238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee94070280 0x7fee941415a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fee8400c570 tx=0x7fee8400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.283+0000 7fee8b400640 1 -- 172.21.15.26:0/1334315238 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee84015070 con 0x7fee94070280 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.283+0000 7fee8b400640 1 -- 172.21.15.26:0/1334315238 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee84011800 con 0x7fee94070280 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.283+0000 7fee8b400640 1 -- 172.21.15.26:0/1334315238 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee84011bf0 con 0x7fee94070280 2024-04-26T21:08:47.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.283+0000 7fee99e00640 1 -- 172.21.15.26:0/1334315238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee94141e60 con 0x7fee94070280 2024-04-26T21:08:47.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.283+0000 7fee99e00640 1 -- 172.21.15.26:0/1334315238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee94142420 con 0x7fee94070280 2024-04-26T21:08:47.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.287+0000 7fee8b400640 1 -- 172.21.15.26:0/1334315238 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee84018070 con 0x7fee94070280 2024-04-26T21:08:47.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.288+0000 7fee8b400640 1 --2- 172.21.15.26:0/1334315238 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fee7406c8f0 0x7fee7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.288+0000 7fee8b400640 1 -- 172.21.15.26:0/1334315238 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fee8408ba40 con 0x7fee94070280 2024-04-26T21:08:47.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.289+0000 7fee92000640 1 --2- 172.21.15.26:0/1334315238 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fee7406c8f0 0x7fee7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.289+0000 7fee99e00640 1 -- 172.21.15.26:0/1334315238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee9406a700 con 0x7fee94070280 2024-04-26T21:08:47.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.289+0000 7fee92000640 1 --2- 172.21.15.26:0/1334315238 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fee7406c8f0 0x7fee7406edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fee941fcc80 tx=0x7fee800079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.294+0000 7f69ba000640 1 -- 172.21.15.26:0/927281906 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4031600 con 0x7f69bc163730 2024-04-26T21:08:47.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.295+0000 7f065be00640 1 -- 172.21.15.26:0/1894633067 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f066415cf60 con 0x7f06640d1b80 2024-04-26T21:08:47.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.294+0000 7f69ba000640 1 -- 172.21.15.26:0/927281906 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69b40317a0 con 0x7f69bc163730 2024-04-26T21:08:47.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.294+0000 7f69ba000640 1 -- 172.21.15.26:0/927281906 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b4031b70 con 0x7f69bc163730 2024-04-26T21:08:47.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.296+0000 7f69c2000640 1 -- 172.21.15.26:0/927281906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69bc163730 msgr2=0x7f69bc0d0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.296+0000 7f69c2000640 1 --2- 172.21.15.26:0/927281906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69bc163730 0x7f69bc0d0410 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0d1920 tx=0x7f69b402a5d0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.296+0000 7f69c2000640 1 -- 172.21.15.26:0/927281906 shutdown_connections 2024-04-26T21:08:47.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.296+0000 7f69c2000640 1 --2- 172.21.15.26:0/927281906 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69bc167180 0x7f69bc0d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.296+0000 7f69c2000640 1 --2- 172.21.15.26:0/927281906 >> v2:172.21.15.26:3300/0 conn(0x7f69bc0b3df0 0x7f69bc0d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.296+0000 7f69c2000640 1 --2- 172.21.15.26:0/927281906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69bc163730 0x7f69bc0d0410 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0d1920 tx=0x7f69b402a5d0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.296+0000 7f69c2000640 1 -- 172.21.15.26:0/927281906 >> 172.21.15.26:0/927281906 conn(0x7f69bc0cd840 msgr2=0x7f69bc0cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:47.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.299+0000 7f69c2000640 1 -- 172.21.15.26:0/927281906 shutdown_connections 2024-04-26T21:08:47.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.299+0000 7f69c2000640 1 -- 172.21.15.26:0/927281906 wait complete. 2024-04-26T21:08:47.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.299+0000 7fee8b400640 1 -- 172.21.15.26:0/1334315238 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fee84059dd0 con 0x7fee94070280 2024-04-26T21:08:47.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.303+0000 7f69c2000640 1 Processor -- start 2024-04-26T21:08:47.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.303+0000 7f69c2000640 1 -- start start 2024-04-26T21:08:47.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.305+0000 7f69c2000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69bc0b3df0 0x7f69bc149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.305+0000 7f69c2000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69bc167180 0x7f69bc0b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.305+0000 7f69c2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69bc0b5760 0x7f69bc1645d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.306+0000 7f69bb400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69bc0b3df0 0x7f69bc149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.306+0000 7f69bb400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69bc0b3df0 0x7f69bc149f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.26:46254/0 (socket says 172.21.15.26:46254) 2024-04-26T21:08:47.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.306+0000 7f69baa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69bc167180 0x7f69bc0b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.306+0000 7f69baa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69bc167180 0x7f69bc0b5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34932/0 (socket says 172.21.15.26:34932) 2024-04-26T21:08:47.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.307+0000 7f69bb400640 1 -- 172.21.15.26:0/1702313076 learned_addr learned my addr 172.21.15.26:0/1702313076 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:47.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.307+0000 7f69c2000640 1 -- 172.21.15.26:0/1702313076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc0d0ad0 con 0x7f69bc167180 2024-04-26T21:08:47.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.307+0000 7f69c2000640 1 -- 172.21.15.26:0/1702313076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc0d0950 con 0x7f69bc0b5760 2024-04-26T21:08:47.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.307+0000 7f69bbe00640 1 --2- 172.21.15.26:0/1702313076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69bc0b5760 0x7f69bc1645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.307+0000 7f69c2000640 1 -- 172.21.15.26:0/1702313076 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f69bc0ce950 con 0x7f69bc0b3df0 2024-04-26T21:08:47.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.309+0000 7f69baa00640 1 -- 172.21.15.26:0/1702313076 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69bc0b3df0 msgr2=0x7f69bc149f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.309+0000 7f69baa00640 1 --2- 172.21.15.26:0/1702313076 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69bc0b3df0 0x7f69bc149f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.309+0000 7f69baa00640 1 -- 172.21.15.26:0/1702313076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69bc0b5760 msgr2=0x7f69bc1645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.309+0000 7f69baa00640 1 --2- 172.21.15.26:0/1702313076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69bc0b5760 0x7f69bc1645d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.309+0000 7f69baa00640 1 -- 172.21.15.26:0/1702313076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69bc156d30 con 0x7f69bc167180 2024-04-26T21:08:47.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.310+0000 7f69baa00640 1 --2- 172.21.15.26:0/1702313076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69bc167180 0x7f69bc0b5220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f69a800e900 tx=0x7f69a800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.311+0000 7f69b3400640 1 -- 172.21.15.26:0/1702313076 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a800ca00 con 0x7f69bc167180 2024-04-26T21:08:47.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.311+0000 7f69b3400640 1 -- 172.21.15.26:0/1702313076 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69a800cba0 con 0x7f69bc167180 2024-04-26T21:08:47.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.311+0000 7f69b3400640 1 -- 172.21.15.26:0/1702313076 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a8004030 con 0x7f69bc167180 2024-04-26T21:08:47.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.311+0000 7f69c2000640 1 -- 172.21.15.26:0/1702313076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69bc157010 con 0x7f69bc167180 2024-04-26T21:08:47.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.311+0000 7f69c2000640 1 -- 172.21.15.26:0/1702313076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69bc1575d0 con 0x7f69bc167180 2024-04-26T21:08:47.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.316+0000 7f69b3400640 1 -- 172.21.15.26:0/1702313076 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f69a80041d0 con 0x7f69bc167180 2024-04-26T21:08:47.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.316+0000 7f69c2000640 1 -- 172.21.15.26:0/1702313076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69bc156ac0 con 0x7f69bc167180 2024-04-26T21:08:47.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.317+0000 7f69b3400640 1 --2- 172.21.15.26:0/1702313076 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f699c06c9c0 0x7f699c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.317+0000 7f69b3400640 1 -- 172.21.15.26:0/1702313076 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f69a808bd90 con 0x7f69bc167180 2024-04-26T21:08:47.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.322+0000 7f69bb400640 1 --2- 172.21.15.26:0/1702313076 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f699c06c9c0 0x7f699c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.327+0000 7f69bb400640 1 --2- 172.21.15.26:0/1702313076 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f699c06c9c0 0x7f699c06ee80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009bd0 tx=0x7f69b402f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.331+0000 7f69b3400640 1 -- 172.21.15.26:0/1702313076 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f69a8017020 con 0x7f69bc167180 2024-04-26T21:08:47.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.587+0000 7ff9ee600640 1 -- 172.21.15.26:0/3278933449 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ff9e00052b0 con 0x7ff9e0108ca0 2024-04-26T21:08:47.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.588+0000 7ff9db400640 1 -- 172.21.15.26:0/3278933449 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff9dc058c60 con 0x7ff9e0108ca0 2024-04-26T21:08:47.589 INFO:teuthology.orchestra.run.smithi026.stdout:55834574855 2024-04-26T21:08:47.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.592+0000 7f0608a00640 1 -- 172.21.15.26:0/2832832722 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f05fc038a10 con 0x7f05fc0530a0 2024-04-26T21:08:47.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.592+0000 7ff9ee600640 1 -- 172.21.15.26:0/3278933449 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff9c806c8f0 msgr2=0x7ff9c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.592+0000 7ff9ee600640 1 --2- 172.21.15.26:0/3278933449 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff9c806c8f0 0x7ff9c806edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4007fd0 tx=0x7ff9d4009040 comp rx=0 tx=0).stop 2024-04-26T21:08:47.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.592+0000 7f05faa00640 1 -- 172.21.15.26:0/2832832722 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f05f407a4a0 con 0x7f05fc0530a0 2024-04-26T21:08:47.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.592+0000 7ff9ee600640 1 -- 172.21.15.26:0/3278933449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9e0108ca0 msgr2=0x7ff9e004e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.592+0000 7ff9ee600640 1 --2- 172.21.15.26:0/3278933449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9e0108ca0 0x7ff9e004e000 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc00c370 tx=0x7ff9dc00c830 comp rx=0 tx=0).stop 2024-04-26T21:08:47.596 INFO:teuthology.orchestra.run.smithi026.stdout:55834574853 2024-04-26T21:08:47.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.593+0000 7ff9ee600640 1 -- 172.21.15.26:0/3278933449 shutdown_connections 2024-04-26T21:08:47.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.593+0000 7ff9ee600640 1 --2- 172.21.15.26:0/3278933449 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff9c806c8f0 0x7ff9c806edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.593+0000 7ff9ee600640 1 --2- 172.21.15.26:0/3278933449 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9e0109110 0x7ff9e004e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.593+0000 7ff9ee600640 1 --2- 172.21.15.26:0/3278933449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9e0108ca0 0x7ff9e004e000 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.593+0000 7ff9ee600640 1 --2- 172.21.15.26:0/3278933449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9e0105160 0x7ff9e00f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.593+0000 7ff9ee600640 1 -- 172.21.15.26:0/3278933449 >> 172.21.15.26:0/3278933449 conn(0x7ff9e0009fa0 msgr2=0x7ff9e00533e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:47.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.593+0000 7ff9ee600640 1 -- 172.21.15.26:0/3278933449 shutdown_connections 2024-04-26T21:08:47.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.593+0000 7ff9ee600640 1 -- 172.21.15.26:0/3278933449 wait complete. 2024-04-26T21:08:47.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.596+0000 7f0608a00640 1 -- 172.21.15.26:0/2832832722 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05e406c8f0 msgr2=0x7f05e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.596+0000 7f0608a00640 1 --2- 172.21.15.26:0/2832832722 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05e406c8f0 0x7f05e406edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0604000b60 tx=0x7f0604068050 comp rx=0 tx=0).stop 2024-04-26T21:08:47.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.596+0000 7f0608a00640 1 -- 172.21.15.26:0/2832832722 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05fc0530a0 msgr2=0x7f05fc0cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.596+0000 7f0608a00640 1 --2- 172.21.15.26:0/2832832722 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05fc0530a0 0x7f05fc0cee50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f05f4031000 tx=0x7f05f40046c0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.596+0000 7f0608a00640 1 -- 172.21.15.26:0/2832832722 shutdown_connections 2024-04-26T21:08:47.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.596+0000 7f0608a00640 1 --2- 172.21.15.26:0/2832832722 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05e406c8f0 0x7f05e406edb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.596+0000 7f0608a00640 1 --2- 172.21.15.26:0/2832832722 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05fc0530a0 0x7f05fc0cee50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.596+0000 7f0608a00640 1 --2- 172.21.15.26:0/2832832722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05fc050180 0x7f05fc04e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.596+0000 7f0608a00640 1 --2- 172.21.15.26:0/2832832722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05fc00aad0 0x7f05fc04b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.597+0000 7f0608a00640 1 -- 172.21.15.26:0/2832832722 >> 172.21.15.26:0/2832832722 conn(0x7f05fc009fb0 msgr2=0x7f05fc036570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:47.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.597+0000 7f0608a00640 1 -- 172.21.15.26:0/2832832722 shutdown_connections 2024-04-26T21:08:47.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.598+0000 7f0608a00640 1 -- 172.21.15.26:0/2832832722 wait complete. 2024-04-26T21:08:47.621 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.7 2024-04-26T21:08:47.621 DEBUG:teuthology.parallel:result is None 2024-04-26T21:08:47.627 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.0 2024-04-26T21:08:47.627 DEBUG:teuthology.parallel:result is None 2024-04-26T21:08:47.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.634+0000 7f1b64800640 1 -- 172.21.15.26:0/275757033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f1b5c06a700 con 0x7f1b5c0d39f0 2024-04-26T21:08:47.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.635+0000 7f1b56000640 1 -- 172.21.15.26:0/275757033 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1b58049030 con 0x7f1b5c0d39f0 2024-04-26T21:08:47.637 INFO:teuthology.orchestra.run.smithi026.stdout:55834574853 2024-04-26T21:08:47.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.638+0000 7f1b64800640 1 -- 172.21.15.26:0/275757033 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1b4006c940 msgr2=0x7f1b4006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.638+0000 7f1b64800640 1 --2- 172.21.15.26:0/275757033 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1b4006c940 0x7f1b4006ee00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c162bf0 tx=0x7f1b4c009040 comp rx=0 tx=0).stop 2024-04-26T21:08:47.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.638+0000 7f1b64800640 1 -- 172.21.15.26:0/275757033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b5c0d39f0 msgr2=0x7f1b5c15a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.638+0000 7f1b64800640 1 --2- 172.21.15.26:0/275757033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b5c0d39f0 0x7f1b5c15a820 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1b5802ad00 tx=0x7f1b58004a40 comp rx=0 tx=0).stop 2024-04-26T21:08:47.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.638+0000 7f1b64800640 1 -- 172.21.15.26:0/275757033 shutdown_connections 2024-04-26T21:08:47.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.638+0000 7f1b64800640 1 --2- 172.21.15.26:0/275757033 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1b4006c940 0x7f1b4006ee00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.638+0000 7f1b64800640 1 --2- 172.21.15.26:0/275757033 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b5c15f710 0x7f1b5c0b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.638+0000 7f1b64800640 1 --2- 172.21.15.26:0/275757033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b5c0d3dd0 0x7f1b5c0ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.638+0000 7f1b64800640 1 --2- 172.21.15.26:0/275757033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b5c0d39f0 0x7f1b5c15a820 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.638+0000 7f1b64800640 1 -- 172.21.15.26:0/275757033 >> 172.21.15.26:0/275757033 conn(0x7f1b5c0d51a0 msgr2=0x7f1b5c06d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:47.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.638+0000 7f1b64800640 1 -- 172.21.15.26:0/275757033 shutdown_connections 2024-04-26T21:08:47.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.639+0000 7f1b64800640 1 -- 172.21.15.26:0/275757033 wait complete. 2024-04-26T21:08:47.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.648+0000 7f1ce7e00640 1 -- 172.21.15.26:0/2956515949 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1cd4005850 con 0x7f1cfc1522f0 2024-04-26T21:08:47.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.649+0000 7f1cf2a00640 1 -- 172.21.15.26:0/2956515949 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1cec07a260 con 0x7f1cfc1522f0 2024-04-26T21:08:47.650 INFO:teuthology.orchestra.run.smithi026.stdout:55834574853 2024-04-26T21:08:47.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.651+0000 7f1ce7e00640 1 -- 172.21.15.26:0/2956515949 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1cdc06caf0 msgr2=0x7f1cdc06efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.651+0000 7f1ce7e00640 1 --2- 172.21.15.26:0/2956515949 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1cdc06caf0 0x7f1cdc06efb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4001d10 tx=0x7f1cf4009040 comp rx=0 tx=0).stop 2024-04-26T21:08:47.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.651+0000 7f1ce7e00640 1 -- 172.21.15.26:0/2956515949 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cfc1522f0 msgr2=0x7f1cfc15a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.651+0000 7f1ce7e00640 1 --2- 172.21.15.26:0/2956515949 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cfc1522f0 0x7f1cfc15a510 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0b0a00 tx=0x7f1cec0046c0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.652+0000 7f1ce7e00640 1 -- 172.21.15.26:0/2956515949 shutdown_connections 2024-04-26T21:08:47.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.652+0000 7f1ce7e00640 1 --2- 172.21.15.26:0/2956515949 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1cdc06caf0 0x7f1cdc06efb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.652+0000 7f1ce7e00640 1 --2- 172.21.15.26:0/2956515949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc15c480 0x7f1cfc205350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.652+0000 7f1ce7e00640 1 --2- 172.21.15.26:0/2956515949 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cfc1522f0 0x7f1cfc15a510 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.652+0000 7f1ce7e00640 1 --2- 172.21.15.26:0/2956515949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cfc0b5ea0 0x7f1cfc1e3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.652+0000 7f1ce7e00640 1 -- 172.21.15.26:0/2956515949 >> 172.21.15.26:0/2956515949 conn(0x7f1cfc06ca30 msgr2=0x7f1cfc0b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:47.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.652+0000 7f1ce7e00640 1 -- 172.21.15.26:0/2956515949 shutdown_connections 2024-04-26T21:08:47.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.652+0000 7f1ce7e00640 1 -- 172.21.15.26:0/2956515949 wait complete. 2024-04-26T21:08:47.664 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.3 2024-04-26T21:08:47.664 DEBUG:teuthology.parallel:result is None 2024-04-26T21:08:47.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.5 2024-04-26T21:08:47.671 DEBUG:teuthology.parallel:result is None 2024-04-26T21:08:47.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.671+0000 7fa9b2e00640 1 -- 172.21.15.26:0/2839877441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa9ac06a700 con 0x7fa9ac0b3df0 2024-04-26T21:08:47.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.671+0000 7fa9a3e00640 1 -- 172.21.15.26:0/2839877441 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa9a4059620 con 0x7fa9ac0b3df0 2024-04-26T21:08:47.672 INFO:teuthology.orchestra.run.smithi026.stdout:55834574853 2024-04-26T21:08:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.674+0000 7fa9b2e00640 1 -- 172.21.15.26:0/2839877441 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa98c06c970 msgr2=0x7fa98c06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.674+0000 7fa9b2e00640 1 --2- 172.21.15.26:0/2839877441 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa98c06c970 0x7fa98c06ee30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0d0280 tx=0x7fa99c008040 comp rx=0 tx=0).stop 2024-04-26T21:08:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.674+0000 7fa9b2e00640 1 -- 172.21.15.26:0/2839877441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9ac0b3df0 msgr2=0x7fa9ac152f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.674+0000 7fa9b2e00640 1 --2- 172.21.15.26:0/2839877441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9ac0b3df0 0x7fa9ac152f50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400ca50 tx=0x7fa9a400cf10 comp rx=0 tx=0).stop 2024-04-26T21:08:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.675+0000 7fa9b2e00640 1 -- 172.21.15.26:0/2839877441 shutdown_connections 2024-04-26T21:08:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.675+0000 7fa9b2e00640 1 --2- 172.21.15.26:0/2839877441 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa98c06c970 0x7fa98c06ee30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.675+0000 7fa9b2e00640 1 --2- 172.21.15.26:0/2839877441 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9ac167180 0x7fa9ac164290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.675+0000 7fa9b2e00640 1 --2- 172.21.15.26:0/2839877441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9ac163730 0x7fa9ac1f9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.675+0000 7fa9b2e00640 1 --2- 172.21.15.26:0/2839877441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9ac0b3df0 0x7fa9ac152f50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.675+0000 7fa9b2e00640 1 -- 172.21.15.26:0/2839877441 >> 172.21.15.26:0/2839877441 conn(0x7fa9ac0cd840 msgr2=0x7fa9ac1616c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.675+0000 7fa9b2e00640 1 -- 172.21.15.26:0/2839877441 shutdown_connections 2024-04-26T21:08:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.675+0000 7fa9b2e00640 1 -- 172.21.15.26:0/2839877441 wait complete. 2024-04-26T21:08:47.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.681+0000 7f066a000640 1 -- 172.21.15.26:0/1894633067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f066415d140 con 0x7f06640d1b80 2024-04-26T21:08:47.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.682+0000 7f065be00640 1 -- 172.21.15.26:0/1894633067 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f064c061020 con 0x7f06640d1b80 2024-04-26T21:08:47.683 INFO:teuthology.orchestra.run.smithi026.stdout:55834574853 2024-04-26T21:08:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.684+0000 7f066a000640 1 -- 172.21.15.26:0/1894633067 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f064006c940 msgr2=0x7f064006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.684+0000 7f066a000640 1 --2- 172.21.15.26:0/1894633067 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f064006c940 0x7f064006ee00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f06640cc080 tx=0x7f065400d040 comp rx=0 tx=0).stop 2024-04-26T21:08:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.684+0000 7f066a000640 1 -- 172.21.15.26:0/1894633067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06640d1b80 msgr2=0x7f066415c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.684+0000 7f066a000640 1 --2- 172.21.15.26:0/1894633067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06640d1b80 0x7f066415c320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f064c00d830 tx=0x7f064c00dcf0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.685+0000 7f066a000640 1 -- 172.21.15.26:0/1894633067 shutdown_connections 2024-04-26T21:08:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.685+0000 7f066a000640 1 --2- 172.21.15.26:0/1894633067 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f064006c940 0x7f064006ee00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.685+0000 7f066a000640 1 --2- 172.21.15.26:0/1894633067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0664160500 0x7f066415dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.685+0000 7f066a000640 1 --2- 172.21.15.26:0/1894633067 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f066415f680 0x7f066415a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.685+0000 7f066a000640 1 --2- 172.21.15.26:0/1894633067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06640d1b80 0x7f066415c320 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.685+0000 7f066a000640 1 -- 172.21.15.26:0/1894633067 >> 172.21.15.26:0/1894633067 conn(0x7f066409e490 msgr2=0x7f06640d51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.685+0000 7f066a000640 1 -- 172.21.15.26:0/1894633067 shutdown_connections 2024-04-26T21:08:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.685+0000 7f066a000640 1 -- 172.21.15.26:0/1894633067 wait complete. 2024-04-26T21:08:47.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.2 2024-04-26T21:08:47.700 DEBUG:teuthology.parallel:result is None 2024-04-26T21:08:47.708 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.6 2024-04-26T21:08:47.708 DEBUG:teuthology.parallel:result is None 2024-04-26T21:08:47.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.717+0000 7fee99e00640 1 -- 172.21.15.26:0/1334315238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fee94070c60 con 0x7fee94070280 2024-04-26T21:08:47.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.718+0000 7fee8b400640 1 -- 172.21.15.26:0/1334315238 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fee840596d0 con 0x7fee94070280 2024-04-26T21:08:47.719 INFO:teuthology.orchestra.run.smithi026.stdout:55834574853 2024-04-26T21:08:47.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.720+0000 7fee99e00640 1 -- 172.21.15.26:0/1334315238 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fee7406c8f0 msgr2=0x7fee7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.720+0000 7fee99e00640 1 --2- 172.21.15.26:0/1334315238 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fee7406c8f0 0x7fee7406edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fee941fcc80 tx=0x7fee800079b0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.720+0000 7fee99e00640 1 -- 172.21.15.26:0/1334315238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee94070280 msgr2=0x7fee941415a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.720+0000 7fee99e00640 1 --2- 172.21.15.26:0/1334315238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee94070280 0x7fee941415a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fee8400c570 tx=0x7fee8400ca30 comp rx=0 tx=0).stop 2024-04-26T21:08:47.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.720+0000 7fee99e00640 1 -- 172.21.15.26:0/1334315238 shutdown_connections 2024-04-26T21:08:47.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.720+0000 7fee99e00640 1 --2- 172.21.15.26:0/1334315238 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fee7406c8f0 0x7fee7406edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.720+0000 7fee99e00640 1 --2- 172.21.15.26:0/1334315238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee940cf6f0 0x7fee94140f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.720+0000 7fee99e00640 1 --2- 172.21.15.26:0/1334315238 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee94071150 0x7fee9415b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.720+0000 7fee99e00640 1 --2- 172.21.15.26:0/1334315238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee94070280 0x7fee941415a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.720+0000 7fee99e00640 1 -- 172.21.15.26:0/1334315238 >> 172.21.15.26:0/1334315238 conn(0x7fee940d1e10 msgr2=0x7fee940b67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:47.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.721+0000 7fee99e00640 1 -- 172.21.15.26:0/1334315238 shutdown_connections 2024-04-26T21:08:47.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.721+0000 7fee99e00640 1 -- 172.21.15.26:0/1334315238 wait complete. 2024-04-26T21:08:47.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.724+0000 7f69c2000640 1 -- 172.21.15.26:0/1702313076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f69bc1577d0 con 0x7f69bc167180 2024-04-26T21:08:47.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.724+0000 7f69b3400640 1 -- 172.21.15.26:0/1702313076 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f69bc1577d0 con 0x7f69bc167180 2024-04-26T21:08:47.725 INFO:teuthology.orchestra.run.smithi026.stdout:55834574854 2024-04-26T21:08:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.726+0000 7f69c2000640 1 -- 172.21.15.26:0/1702313076 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f699c06c9c0 msgr2=0x7f699c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.726+0000 7f69c2000640 1 --2- 172.21.15.26:0/1702313076 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f699c06c9c0 0x7f699c06ee80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009bd0 tx=0x7f69b402f000 comp rx=0 tx=0).stop 2024-04-26T21:08:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.726+0000 7f69c2000640 1 -- 172.21.15.26:0/1702313076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69bc167180 msgr2=0x7f69bc0b5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.726+0000 7f69c2000640 1 --2- 172.21.15.26:0/1702313076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69bc167180 0x7f69bc0b5220 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f69a800e900 tx=0x7f69a800edc0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.726+0000 7f69c2000640 1 -- 172.21.15.26:0/1702313076 shutdown_connections 2024-04-26T21:08:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.726+0000 7f69c2000640 1 --2- 172.21.15.26:0/1702313076 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f699c06c9c0 0x7f699c06ee80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.726+0000 7f69c2000640 1 --2- 172.21.15.26:0/1702313076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69bc0b5760 0x7f69bc1645d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.726+0000 7f69c2000640 1 --2- 172.21.15.26:0/1702313076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69bc167180 0x7f69bc0b5220 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.726+0000 7f69c2000640 1 --2- 172.21.15.26:0/1702313076 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69bc0b3df0 0x7f69bc149f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.726+0000 7f69c2000640 1 -- 172.21.15.26:0/1702313076 >> 172.21.15.26:0/1702313076 conn(0x7f69bc0cd840 msgr2=0x7f69bc0b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:47.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.726+0000 7f69c2000640 1 -- 172.21.15.26:0/1702313076 shutdown_connections 2024-04-26T21:08:47.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.726+0000 7f69c2000640 1 -- 172.21.15.26:0/1702313076 wait complete. 2024-04-26T21:08:47.739 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.4 2024-04-26T21:08:47.739 DEBUG:teuthology.parallel:result is None 2024-04-26T21:08:47.742 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.1 2024-04-26T21:08:47.743 DEBUG:teuthology.parallel:result is None 2024-04-26T21:08:47.743 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T21:08:47.743 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:08:47.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.991+0000 7ff133600640 1 Processor -- start 2024-04-26T21:08:47.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.991+0000 7ff133600640 1 -- start start 2024-04-26T21:08:47.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.992+0000 7ff133600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff12c0d39f0 0x7ff12c0d5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.993+0000 7ff133600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff12c15f710 0x7ff12c0d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.993+0000 7ff133600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff12c0d0ef0 con 0x7ff12c0d3dd0 2024-04-26T21:08:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.993+0000 7ff133600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ff12c0b4900 con 0x7ff12c0d39f0 2024-04-26T21:08:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.993+0000 7ff133600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff12c0b4be0 con 0x7ff12c15f710 2024-04-26T21:08:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.993+0000 7ff12be00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff12c0d3dd0 0x7ff12c15f000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44888/0 (socket says 172.21.15.26:44888) 2024-04-26T21:08:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.993+0000 7ff12be00640 1 -- 172.21.15.26:0/3227775510 learned_addr learned my addr 172.21.15.26:0/3227775510 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.993+0000 7ff130c00640 1 --2- 172.21.15.26:0/3227775510 >> v2:172.21.15.26:3300/0 conn(0x7ff12c0d39f0 0x7ff12c0d5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.993+0000 7ff131600640 1 --2- 172.21.15.26:0/3227775510 >> v2:172.21.15.86:3300/0 conn(0x7ff12c15f710 0x7ff12c0d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.994+0000 7ff12b400640 1 -- 172.21.15.26:0/3227775510 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4242200103 0 0) 0x7ff12c0d0ef0 con 0x7ff12c0d3dd0 2024-04-26T21:08:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.994+0000 7ff12b400640 1 -- 172.21.15.26:0/3227775510 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff114003520 con 0x7ff12c0d3dd0 2024-04-26T21:08:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.994+0000 7ff130c00640 1 -- 172.21.15.26:0/3227775510 >> v2:172.21.15.86:3300/0 conn(0x7ff12c15f710 msgr2=0x7ff12c0d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.994+0000 7ff130c00640 1 --2- 172.21.15.26:0/3227775510 >> v2:172.21.15.86:3300/0 conn(0x7ff12c15f710 0x7ff12c0d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.994+0000 7ff130c00640 1 -- 172.21.15.26:0/3227775510 >> v1:172.21.15.26:6789/0 conn(0x7ff12c0d3dd0 legacy=0x7ff12c15f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.994+0000 7ff130c00640 1 -- 172.21.15.26:0/3227775510 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12c0b4ec0 con 0x7ff12c0d39f0 2024-04-26T21:08:47.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.995+0000 7ff130c00640 1 --2- 172.21.15.26:0/3227775510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff12c0d39f0 0x7ff12c0d5cc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0d4240 tx=0x7ff12002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6e0b57259ed8c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.995+0000 7ff12b400640 1 -- 172.21.15.26:0/3227775510 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120037070 con 0x7ff12c0d39f0 2024-04-26T21:08:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.995+0000 7ff12b400640 1 -- 172.21.15.26:0/3227775510 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff120032440 con 0x7ff12c0d39f0 2024-04-26T21:08:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.995+0000 7ff12b400640 1 -- 172.21.15.26:0/3227775510 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1200327f0 con 0x7ff12c0d39f0 2024-04-26T21:08:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.996+0000 7ff133600640 1 -- 172.21.15.26:0/3227775510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff12c0d39f0 msgr2=0x7ff12c0d5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.996+0000 7ff133600640 1 --2- 172.21.15.26:0/3227775510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff12c0d39f0 0x7ff12c0d5cc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0d4240 tx=0x7ff12002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.996+0000 7ff133600640 1 -- 172.21.15.26:0/3227775510 shutdown_connections 2024-04-26T21:08:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.996+0000 7ff133600640 1 --2- 172.21.15.26:0/3227775510 >> v2:172.21.15.86:3300/0 conn(0x7ff12c15f710 0x7ff12c0d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.996+0000 7ff133600640 1 --2- 172.21.15.26:0/3227775510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff12c0d39f0 0x7ff12c0d5cc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.996+0000 7ff133600640 1 -- 172.21.15.26:0/3227775510 >> 172.21.15.26:0/3227775510 conn(0x7ff12c0d51a0 msgr2=0x7ff12c0d55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.996+0000 7ff133600640 1 -- 172.21.15.26:0/3227775510 shutdown_connections 2024-04-26T21:08:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.996+0000 7ff133600640 1 -- 172.21.15.26:0/3227775510 wait complete. 2024-04-26T21:08:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.997+0000 7ff133600640 1 Processor -- start 2024-04-26T21:08:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.998+0000 7ff133600640 1 -- start start 2024-04-26T21:08:47.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.998+0000 7ff133600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff12c0d39f0 0x7ff12c0c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.998+0000 7ff133600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff12c0d3dd0 0x7ff12c155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:47.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.998+0000 7ff133600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff12c15f710 0x7ff12c160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.998+0000 7ff133600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c0b5770 con 0x7ff12c0d39f0 2024-04-26T21:08:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.998+0000 7ff133600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c0b55f0 con 0x7ff12c0d3dd0 2024-04-26T21:08:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.998+0000 7ff133600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff12c0b58f0 con 0x7ff12c15f710 2024-04-26T21:08:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.999+0000 7ff130c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff12c0d39f0 0x7ff12c0c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.999+0000 7ff130c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff12c0d39f0 0x7ff12c0c8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34948/0 (socket says 172.21.15.26:34948) 2024-04-26T21:08:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.999+0000 7ff130c00640 1 -- 172.21.15.26:0/347774496 learned_addr learned my addr 172.21.15.26:0/347774496 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.999+0000 7ff130c00640 1 -- 172.21.15.26:0/347774496 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff12c15f710 msgr2=0x7ff12c160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.999+0000 7ff130c00640 1 --2- 172.21.15.26:0/347774496 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff12c15f710 0x7ff12c160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.999+0000 7ff130c00640 1 -- 172.21.15.26:0/347774496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff12c0d3dd0 msgr2=0x7ff12c155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.999+0000 7ff130c00640 1 --2- 172.21.15.26:0/347774496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff12c0d3dd0 0x7ff12c155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.999+0000 7ff130c00640 1 -- 172.21.15.26:0/347774496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12c0cbbb0 con 0x7ff12c0d39f0 2024-04-26T21:08:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:47.999+0000 7ff130c00640 1 --2- 172.21.15.26:0/347774496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff12c0d39f0 0x7ff12c0c8ec0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff12002a1e0 tx=0x7ff1200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.000+0000 7ff128c00640 1 -- 172.21.15.26:0/347774496 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120041030 con 0x7ff12c0d39f0 2024-04-26T21:08:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.000+0000 7ff128c00640 1 -- 172.21.15.26:0/347774496 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff12003d420 con 0x7ff12c0d39f0 2024-04-26T21:08:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.000+0000 7ff128c00640 1 -- 172.21.15.26:0/347774496 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120037040 con 0x7ff12c0d39f0 2024-04-26T21:08:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.000+0000 7ff133600640 1 -- 172.21.15.26:0/347774496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12c0cbe90 con 0x7ff12c0d39f0 2024-04-26T21:08:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.000+0000 7ff133600640 1 -- 172.21.15.26:0/347774496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff12c0b88a0 con 0x7ff12c0d39f0 2024-04-26T21:08:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.001+0000 7ff133600640 1 -- 172.21.15.26:0/347774496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff12c06a700 con 0x7ff12c0d39f0 2024-04-26T21:08:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.002+0000 7ff128c00640 1 -- 172.21.15.26:0/347774496 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff120044070 con 0x7ff12c0d39f0 2024-04-26T21:08:48.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.003+0000 7ff128c00640 1 --2- 172.21.15.26:0/347774496 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff10c06c8f0 0x7ff10c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:48.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.003+0000 7ff128c00640 1 -- 172.21.15.26:0/347774496 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff1200ac2b0 con 0x7ff12c0d39f0 2024-04-26T21:08:48.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.003+0000 7ff12be00640 1 --2- 172.21.15.26:0/347774496 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff10c06c8f0 0x7ff10c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:48.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.004+0000 7ff12be00640 1 --2- 172.21.15.26:0/347774496 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff10c06c8f0 0x7ff10c06edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff118000f60 tx=0x7ff1180078f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:48.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.007+0000 7ff128c00640 1 -- 172.21.15.26:0/347774496 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff12007a5c0 con 0x7ff12c0d39f0 2024-04-26T21:08:48.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.199+0000 7ff133600640 1 -- 172.21.15.26:0/347774496 --> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff12c0b8ae0 con 0x7ff10c06c8f0 2024-04-26T21:08:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.219+0000 7ff128c00640 1 -- 172.21.15.26:0/347774496 <== mgr.4104 v2:172.21.15.26:6832/3130430358 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7ff118005890 con 0x7ff10c06c8f0 2024-04-26T21:08:48.223 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-26T21:08:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.224+0000 7ff133600640 1 -- 172.21.15.26:0/347774496 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff10c06c8f0 msgr2=0x7ff10c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.224+0000 7ff133600640 1 --2- 172.21.15.26:0/347774496 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff10c06c8f0 0x7ff10c06edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff118000f60 tx=0x7ff1180078f0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.224+0000 7ff133600640 1 -- 172.21.15.26:0/347774496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff12c0d39f0 msgr2=0x7ff12c0c8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.224+0000 7ff133600640 1 --2- 172.21.15.26:0/347774496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff12c0d39f0 0x7ff12c0c8ec0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff12002a1e0 tx=0x7ff1200046c0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.224+0000 7ff133600640 1 -- 172.21.15.26:0/347774496 shutdown_connections 2024-04-26T21:08:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.224+0000 7ff133600640 1 --2- 172.21.15.26:0/347774496 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff10c06c8f0 0x7ff10c06edb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.224+0000 7ff133600640 1 --2- 172.21.15.26:0/347774496 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff12c15f710 0x7ff12c160bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.224+0000 7ff133600640 1 --2- 172.21.15.26:0/347774496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff12c0d3dd0 0x7ff12c155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.224+0000 7ff133600640 1 --2- 172.21.15.26:0/347774496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff12c0d39f0 0x7ff12c0c8ec0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.224+0000 7ff133600640 1 -- 172.21.15.26:0/347774496 >> 172.21.15.26:0/347774496 conn(0x7ff12c0d51a0 msgr2=0x7ff12c0b5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.224+0000 7ff133600640 1 -- 172.21.15.26:0/347774496 shutdown_connections 2024-04-26T21:08:48.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.224+0000 7ff133600640 1 -- 172.21.15.26:0/347774496 wait complete. 2024-04-26T21:08:48.243 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-26T21:08:47.292515+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235764,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738956,"statfs":{"total":773094113280,"available":772852690944,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.131669"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711962+0000","last_change":"2024-04-26T21:08:38.198898+0000","last_active":"2024-04-26T21:08:39.711962+0000","last_peered":"2024-04-26T21:08:39.711962+0000","last_clean":"2024-04-26T21:08:39.711962+0000","last_became_active":"2024-04-26T21:08:38.198596+0000","last_became_peered":"2024-04-26T21:08:38.198596+0000","last_unstale":"2024-04-26T21:08:39.711962+0000","last_undegraded":"2024-04-26T21:08:39.711962+0000","last_fullsized":"2024-04-26T21:08:39.711962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:19.756896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211160+0000","last_change":"2024-04-26T21:08:38.194080+0000","last_active":"2024-04-26T21:08:39.211160+0000","last_peered":"2024-04-26T21:08:39.211160+0000","last_clean":"2024-04-26T21:08:39.211160+0000","last_became_active":"2024-04-26T21:08:38.193534+0000","last_became_peered":"2024-04-26T21:08:38.193534+0000","last_unstale":"2024-04-26T21:08:39.211160+0000","last_undegraded":"2024-04-26T21:08:39.211160+0000","last_fullsized":"2024-04-26T21:08:39.211160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:22.132136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712041+0000","last_change":"2024-04-26T21:08:38.198384+0000","last_active":"2024-04-26T21:08:39.712041+0000","last_peered":"2024-04-26T21:08:39.712041+0000","last_clean":"2024-04-26T21:08:39.712041+0000","last_became_active":"2024-04-26T21:08:38.198055+0000","last_became_peered":"2024-04-26T21:08:38.198055+0000","last_unstale":"2024-04-26T21:08:39.712041+0000","last_undegraded":"2024-04-26T21:08:39.712041+0000","last_fullsized":"2024-04-26T21:08:39.712041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:55.566474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194277+0000","last_change":"2024-04-26T21:08:38.205534+0000","last_active":"2024-04-26T21:08:39.194277+0000","last_peered":"2024-04-26T21:08:39.194277+0000","last_clean":"2024-04-26T21:08:39.194277+0000","last_became_active":"2024-04-26T21:08:38.205235+0000","last_became_peered":"2024-04-26T21:08:38.205235+0000","last_unstale":"2024-04-26T21:08:39.194277+0000","last_undegraded":"2024-04-26T21:08:39.194277+0000","last_fullsized":"2024-04-26T21:08:39.194277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:26.983477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711420+0000","last_change":"2024-04-26T21:08:38.200597+0000","last_active":"2024-04-26T21:08:39.711420+0000","last_peered":"2024-04-26T21:08:39.711420+0000","last_clean":"2024-04-26T21:08:39.711420+0000","last_became_active":"2024-04-26T21:08:38.197772+0000","last_became_peered":"2024-04-26T21:08:38.197772+0000","last_unstale":"2024-04-26T21:08:39.711420+0000","last_undegraded":"2024-04-26T21:08:39.711420+0000","last_fullsized":"2024-04-26T21:08:39.711420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:17.370683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.196006+0000","last_change":"2024-04-26T21:08:38.210251+0000","last_active":"2024-04-26T21:08:39.196006+0000","last_peered":"2024-04-26T21:08:39.196006+0000","last_clean":"2024-04-26T21:08:39.196006+0000","last_became_active":"2024-04-26T21:08:38.209628+0000","last_became_peered":"2024-04-26T21:08:38.209628+0000","last_unstale":"2024-04-26T21:08:39.196006+0000","last_undegraded":"2024-04-26T21:08:39.196006+0000","last_fullsized":"2024-04-26T21:08:39.196006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:28.069603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711557+0000","last_change":"2024-04-26T21:08:38.199522+0000","last_active":"2024-04-26T21:08:39.711557+0000","last_peered":"2024-04-26T21:08:39.711557+0000","last_clean":"2024-04-26T21:08:39.711557+0000","last_became_active":"2024-04-26T21:08:38.198122+0000","last_became_peered":"2024-04-26T21:08:38.198122+0000","last_unstale":"2024-04-26T21:08:39.711557+0000","last_undegraded":"2024-04-26T21:08:39.711557+0000","last_fullsized":"2024-04-26T21:08:39.711557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:55.797399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711087+0000","last_change":"2024-04-26T21:08:38.207966+0000","last_active":"2024-04-26T21:08:39.711087+0000","last_peered":"2024-04-26T21:08:39.711087+0000","last_clean":"2024-04-26T21:08:39.711087+0000","last_became_active":"2024-04-26T21:08:38.207617+0000","last_became_peered":"2024-04-26T21:08:38.207617+0000","last_unstale":"2024-04-26T21:08:39.711087+0000","last_undegraded":"2024-04-26T21:08:39.711087+0000","last_fullsized":"2024-04-26T21:08:39.711087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:55.566474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712128+0000","last_change":"2024-04-26T21:08:38.204121+0000","last_active":"2024-04-26T21:08:39.712128+0000","last_peered":"2024-04-26T21:08:39.712128+0000","last_clean":"2024-04-26T21:08:39.712128+0000","last_became_active":"2024-04-26T21:08:38.203125+0000","last_became_peered":"2024-04-26T21:08:38.203125+0000","last_unstale":"2024-04-26T21:08:39.712128+0000","last_undegraded":"2024-04-26T21:08:39.712128+0000","last_fullsized":"2024-04-26T21:08:39.712128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:36.343803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217352+0000","last_change":"2024-04-26T21:08:38.204308+0000","last_active":"2024-04-26T21:08:39.217352+0000","last_peered":"2024-04-26T21:08:39.217352+0000","last_clean":"2024-04-26T21:08:39.217352+0000","last_became_active":"2024-04-26T21:08:38.203942+0000","last_became_peered":"2024-04-26T21:08:38.203942+0000","last_unstale":"2024-04-26T21:08:39.217352+0000","last_undegraded":"2024-04-26T21:08:39.217352+0000","last_fullsized":"2024-04-26T21:08:39.217352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:42.506444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217214+0000","last_change":"2024-04-26T21:08:38.201929+0000","last_active":"2024-04-26T21:08:39.217214+0000","last_peered":"2024-04-26T21:08:39.217214+0000","last_clean":"2024-04-26T21:08:39.217214+0000","last_became_active":"2024-04-26T21:08:38.201646+0000","last_became_peered":"2024-04-26T21:08:38.201646+0000","last_unstale":"2024-04-26T21:08:39.217214+0000","last_undegraded":"2024-04-26T21:08:39.217214+0000","last_fullsized":"2024-04-26T21:08:39.217214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:40.618646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195574+0000","last_change":"2024-04-26T21:08:38.207847+0000","last_active":"2024-04-26T21:08:39.195574+0000","last_peered":"2024-04-26T21:08:39.195574+0000","last_clean":"2024-04-26T21:08:39.195574+0000","last_became_active":"2024-04-26T21:08:38.204908+0000","last_became_peered":"2024-04-26T21:08:38.204908+0000","last_unstale":"2024-04-26T21:08:39.195574+0000","last_undegraded":"2024-04-26T21:08:39.195574+0000","last_fullsized":"2024-04-26T21:08:39.195574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:00.992973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711183+0000","last_change":"2024-04-26T21:08:38.203775+0000","last_active":"2024-04-26T21:08:39.711183+0000","last_peered":"2024-04-26T21:08:39.711183+0000","last_clean":"2024-04-26T21:08:39.711183+0000","last_became_active":"2024-04-26T21:08:38.203139+0000","last_became_peered":"2024-04-26T21:08:38.203139+0000","last_unstale":"2024-04-26T21:08:39.711183+0000","last_undegraded":"2024-04-26T21:08:39.711183+0000","last_fullsized":"2024-04-26T21:08:39.711183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:01.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712480+0000","last_change":"2024-04-26T21:08:38.195656+0000","last_active":"2024-04-26T21:08:39.712480+0000","last_peered":"2024-04-26T21:08:39.712480+0000","last_clean":"2024-04-26T21:08:39.712480+0000","last_became_active":"2024-04-26T21:08:38.195240+0000","last_became_peered":"2024-04-26T21:08:38.195240+0000","last_unstale":"2024-04-26T21:08:39.712480+0000","last_undegraded":"2024-04-26T21:08:39.712480+0000","last_fullsized":"2024-04-26T21:08:39.712480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:16.232014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211031+0000","last_change":"2024-04-26T21:08:38.186523+0000","last_active":"2024-04-26T21:08:39.211031+0000","last_peered":"2024-04-26T21:08:39.211031+0000","last_clean":"2024-04-26T21:08:39.211031+0000","last_became_active":"2024-04-26T21:08:38.186159+0000","last_became_peered":"2024-04-26T21:08:38.186159+0000","last_unstale":"2024-04-26T21:08:39.211031+0000","last_undegraded":"2024-04-26T21:08:39.211031+0000","last_fullsized":"2024-04-26T21:08:39.211031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:40.618646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194033+0000","last_change":"2024-04-26T21:08:38.212958+0000","last_active":"2024-04-26T21:08:39.194033+0000","last_peered":"2024-04-26T21:08:39.194033+0000","last_clean":"2024-04-26T21:08:39.194033+0000","last_became_active":"2024-04-26T21:08:38.212666+0000","last_became_peered":"2024-04-26T21:08:38.212666+0000","last_unstale":"2024-04-26T21:08:39.194033+0000","last_undegraded":"2024-04-26T21:08:39.194033+0000","last_fullsized":"2024-04-26T21:08:39.194033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:42.506444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195541+0000","last_change":"2024-04-26T21:08:38.214751+0000","last_active":"2024-04-26T21:08:39.195541+0000","last_peered":"2024-04-26T21:08:39.195541+0000","last_clean":"2024-04-26T21:08:39.195541+0000","last_became_active":"2024-04-26T21:08:38.212857+0000","last_became_peered":"2024-04-26T21:08:38.212857+0000","last_unstale":"2024-04-26T21:08:39.195541+0000","last_undegraded":"2024-04-26T21:08:39.195541+0000","last_fullsized":"2024-04-26T21:08:39.195541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:35.132498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.191204+0000","last_change":"2024-04-26T21:08:38.198823+0000","last_active":"2024-04-26T21:08:39.191204+0000","last_peered":"2024-04-26T21:08:39.191204+0000","last_clean":"2024-04-26T21:08:39.191204+0000","last_became_active":"2024-04-26T21:08:38.198521+0000","last_became_peered":"2024-04-26T21:08:38.198521+0000","last_unstale":"2024-04-26T21:08:39.191204+0000","last_undegraded":"2024-04-26T21:08:39.191204+0000","last_fullsized":"2024-04-26T21:08:39.191204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:34.390026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712075+0000","last_change":"2024-04-26T21:08:38.200413+0000","last_active":"2024-04-26T21:08:39.712075+0000","last_peered":"2024-04-26T21:08:39.712075+0000","last_clean":"2024-04-26T21:08:39.712075+0000","last_became_active":"2024-04-26T21:08:38.200023+0000","last_became_peered":"2024-04-26T21:08:38.200023+0000","last_unstale":"2024-04-26T21:08:39.712075+0000","last_undegraded":"2024-04-26T21:08:39.712075+0000","last_fullsized":"2024-04-26T21:08:39.712075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:28.069603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.216976+0000","last_change":"2024-04-26T21:08:38.211354+0000","last_active":"2024-04-26T21:08:39.216976+0000","last_peered":"2024-04-26T21:08:39.216976+0000","last_clean":"2024-04-26T21:08:39.216976+0000","last_became_active":"2024-04-26T21:08:38.211016+0000","last_became_peered":"2024-04-26T21:08:38.211016+0000","last_unstale":"2024-04-26T21:08:39.216976+0000","last_undegraded":"2024-04-26T21:08:39.216976+0000","last_fullsized":"2024-04-26T21:08:39.216976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:29.659217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712867+0000","last_change":"2024-04-26T21:08:38.201685+0000","last_active":"2024-04-26T21:08:39.712867+0000","last_peered":"2024-04-26T21:08:39.712867+0000","last_clean":"2024-04-26T21:08:39.712867+0000","last_became_active":"2024-04-26T21:08:38.200467+0000","last_became_peered":"2024-04-26T21:08:38.200467+0000","last_unstale":"2024-04-26T21:08:39.712867+0000","last_undegraded":"2024-04-26T21:08:39.712867+0000","last_fullsized":"2024-04-26T21:08:39.712867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:40.618646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194361+0000","last_change":"2024-04-26T21:08:38.204033+0000","last_active":"2024-04-26T21:08:39.194361+0000","last_peered":"2024-04-26T21:08:39.194361+0000","last_clean":"2024-04-26T21:08:39.194361+0000","last_became_active":"2024-04-26T21:08:38.202425+0000","last_became_peered":"2024-04-26T21:08:38.202425+0000","last_unstale":"2024-04-26T21:08:39.194361+0000","last_undegraded":"2024-04-26T21:08:39.194361+0000","last_fullsized":"2024-04-26T21:08:39.194361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:00.305226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711233+0000","last_change":"2024-04-26T21:08:38.216188+0000","last_active":"2024-04-26T21:08:39.711233+0000","last_peered":"2024-04-26T21:08:39.711233+0000","last_clean":"2024-04-26T21:08:39.711233+0000","last_became_active":"2024-04-26T21:08:38.215866+0000","last_became_peered":"2024-04-26T21:08:38.215866+0000","last_unstale":"2024-04-26T21:08:39.711233+0000","last_undegraded":"2024-04-26T21:08:39.711233+0000","last_fullsized":"2024-04-26T21:08:39.711233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:29.659217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.193946+0000","last_change":"2024-04-26T21:08:38.213795+0000","last_active":"2024-04-26T21:08:39.193946+0000","last_peered":"2024-04-26T21:08:39.193946+0000","last_clean":"2024-04-26T21:08:39.193946+0000","last_became_active":"2024-04-26T21:08:38.211952+0000","last_became_peered":"2024-04-26T21:08:38.211952+0000","last_unstale":"2024-04-26T21:08:39.193946+0000","last_undegraded":"2024-04-26T21:08:39.193946+0000","last_fullsized":"2024-04-26T21:08:39.193946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:22.132136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712055+0000","last_change":"2024-04-26T21:08:38.211961+0000","last_active":"2024-04-26T21:08:39.712055+0000","last_peered":"2024-04-26T21:08:39.712055+0000","last_clean":"2024-04-26T21:08:39.712055+0000","last_became_active":"2024-04-26T21:08:38.211590+0000","last_became_peered":"2024-04-26T21:08:38.211590+0000","last_unstale":"2024-04-26T21:08:39.712055+0000","last_undegraded":"2024-04-26T21:08:39.712055+0000","last_fullsized":"2024-04-26T21:08:39.712055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:22.132136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217302+0000","last_change":"2024-04-26T21:08:38.200054+0000","last_active":"2024-04-26T21:08:39.217302+0000","last_peered":"2024-04-26T21:08:39.217302+0000","last_clean":"2024-04-26T21:08:39.217302+0000","last_became_active":"2024-04-26T21:08:38.199738+0000","last_became_peered":"2024-04-26T21:08:38.199738+0000","last_unstale":"2024-04-26T21:08:39.217302+0000","last_undegraded":"2024-04-26T21:08:39.217302+0000","last_fullsized":"2024-04-26T21:08:39.217302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:22.132136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.210875+0000","last_change":"2024-04-26T21:08:38.195577+0000","last_active":"2024-04-26T21:08:39.210875+0000","last_peered":"2024-04-26T21:08:39.210875+0000","last_clean":"2024-04-26T21:08:39.210875+0000","last_became_active":"2024-04-26T21:08:38.195284+0000","last_became_peered":"2024-04-26T21:08:38.195284+0000","last_unstale":"2024-04-26T21:08:39.210875+0000","last_undegraded":"2024-04-26T21:08:39.210875+0000","last_fullsized":"2024-04-26T21:08:39.210875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:29.659217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.713361+0000","last_change":"2024-04-26T21:08:38.202932+0000","last_active":"2024-04-26T21:08:39.713361+0000","last_peered":"2024-04-26T21:08:39.713361+0000","last_clean":"2024-04-26T21:08:39.713361+0000","last_became_active":"2024-04-26T21:08:38.202614+0000","last_became_peered":"2024-04-26T21:08:38.202614+0000","last_unstale":"2024-04-26T21:08:39.713361+0000","last_undegraded":"2024-04-26T21:08:39.713361+0000","last_fullsized":"2024-04-26T21:08:39.713361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:44.324217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.713283+0000","last_change":"2024-04-26T21:08:38.198429+0000","last_active":"2024-04-26T21:08:39.713283+0000","last_peered":"2024-04-26T21:08:39.713283+0000","last_clean":"2024-04-26T21:08:39.713283+0000","last_became_active":"2024-04-26T21:08:38.198144+0000","last_became_peered":"2024-04-26T21:08:38.198144+0000","last_unstale":"2024-04-26T21:08:39.713283+0000","last_undegraded":"2024-04-26T21:08:39.713283+0000","last_fullsized":"2024-04-26T21:08:39.713283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:22.132136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194610+0000","last_change":"2024-04-26T21:08:38.205890+0000","last_active":"2024-04-26T21:08:39.194610+0000","last_peered":"2024-04-26T21:08:39.194610+0000","last_clean":"2024-04-26T21:08:39.194610+0000","last_became_active":"2024-04-26T21:08:38.205583+0000","last_became_peered":"2024-04-26T21:08:38.205583+0000","last_unstale":"2024-04-26T21:08:39.194610+0000","last_undegraded":"2024-04-26T21:08:39.194610+0000","last_fullsized":"2024-04-26T21:08:39.194610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:17.370683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.190845+0000","last_change":"2024-04-26T21:08:38.204162+0000","last_active":"2024-04-26T21:08:39.190845+0000","last_peered":"2024-04-26T21:08:39.190845+0000","last_clean":"2024-04-26T21:08:39.190845+0000","last_became_active":"2024-04-26T21:08:38.203855+0000","last_became_peered":"2024-04-26T21:08:38.203855+0000","last_unstale":"2024-04-26T21:08:39.190845+0000","last_undegraded":"2024-04-26T21:08:39.190845+0000","last_fullsized":"2024-04-26T21:08:39.190845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:31.643503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.216694+0000","last_change":"2024-04-26T21:08:38.207713+0000","last_active":"2024-04-26T21:08:39.216694+0000","last_peered":"2024-04-26T21:08:39.216694+0000","last_clean":"2024-04-26T21:08:39.216694+0000","last_became_active":"2024-04-26T21:08:38.207384+0000","last_became_peered":"2024-04-26T21:08:38.207384+0000","last_unstale":"2024-04-26T21:08:39.216694+0000","last_undegraded":"2024-04-26T21:08:39.216694+0000","last_fullsized":"2024-04-26T21:08:39.216694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:55.566474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.196940+0000","last_change":"2024-04-26T21:08:38.194999+0000","last_active":"2024-04-26T21:08:39.196940+0000","last_peered":"2024-04-26T21:08:39.196940+0000","last_clean":"2024-04-26T21:08:39.196940+0000","last_became_active":"2024-04-26T21:08:38.194647+0000","last_became_peered":"2024-04-26T21:08:38.194647+0000","last_unstale":"2024-04-26T21:08:39.196940+0000","last_undegraded":"2024-04-26T21:08:39.196940+0000","last_fullsized":"2024-04-26T21:08:39.196940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:34.390026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711906+0000","last_change":"2024-04-26T21:08:37.168719+0000","last_active":"2024-04-26T21:08:39.711906+0000","last_peered":"2024-04-26T21:08:39.711906+0000","last_clean":"2024-04-26T21:08:39.711906+0000","last_became_active":"2024-04-26T21:08:37.168257+0000","last_became_peered":"2024-04-26T21:08:37.168257+0000","last_unstale":"2024-04-26T21:08:39.711906+0000","last_undegraded":"2024-04-26T21:08:39.711906+0000","last_fullsized":"2024-04-26T21:08:39.711906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:18.839848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.713404+0000","last_change":"2024-04-26T21:08:38.190358+0000","last_active":"2024-04-26T21:08:39.713404+0000","last_peered":"2024-04-26T21:08:39.713404+0000","last_clean":"2024-04-26T21:08:39.713404+0000","last_became_active":"2024-04-26T21:08:38.189905+0000","last_became_peered":"2024-04-26T21:08:38.189905+0000","last_unstale":"2024-04-26T21:08:39.713404+0000","last_undegraded":"2024-04-26T21:08:39.713404+0000","last_fullsized":"2024-04-26T21:08:39.713404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:42.506444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.216776+0000","last_change":"2024-04-26T21:08:37.177780+0000","last_active":"2024-04-26T21:08:39.216776+0000","last_peered":"2024-04-26T21:08:39.216776+0000","last_clean":"2024-04-26T21:08:39.216776+0000","last_became_active":"2024-04-26T21:08:37.177488+0000","last_became_peered":"2024-04-26T21:08:37.177488+0000","last_unstale":"2024-04-26T21:08:39.216776+0000","last_undegraded":"2024-04-26T21:08:39.216776+0000","last_fullsized":"2024-04-26T21:08:39.216776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:59.977930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711590+0000","last_change":"2024-04-26T21:08:37.171155+0000","last_active":"2024-04-26T21:08:39.711590+0000","last_peered":"2024-04-26T21:08:39.711590+0000","last_clean":"2024-04-26T21:08:39.711590+0000","last_became_active":"2024-04-26T21:08:37.170816+0000","last_became_peered":"2024-04-26T21:08:37.170816+0000","last_unstale":"2024-04-26T21:08:39.711590+0000","last_undegraded":"2024-04-26T21:08:39.711590+0000","last_fullsized":"2024-04-26T21:08:39.711590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:17.499100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712682+0000","last_change":"2024-04-26T21:08:38.206720+0000","last_active":"2024-04-26T21:08:39.712682+0000","last_peered":"2024-04-26T21:08:39.712682+0000","last_clean":"2024-04-26T21:08:39.712682+0000","last_became_active":"2024-04-26T21:08:38.206401+0000","last_became_peered":"2024-04-26T21:08:38.206401+0000","last_unstale":"2024-04-26T21:08:39.712682+0000","last_undegraded":"2024-04-26T21:08:39.712682+0000","last_fullsized":"2024-04-26T21:08:39.712682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:00.305226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194170+0000","last_change":"2024-04-26T21:08:37.161426+0000","last_active":"2024-04-26T21:08:39.194170+0000","last_peered":"2024-04-26T21:08:39.194170+0000","last_clean":"2024-04-26T21:08:39.194170+0000","last_became_active":"2024-04-26T21:08:37.161119+0000","last_became_peered":"2024-04-26T21:08:37.161119+0000","last_unstale":"2024-04-26T21:08:39.194170+0000","last_undegraded":"2024-04-26T21:08:39.194170+0000","last_fullsized":"2024-04-26T21:08:39.194170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:34.683220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211320+0000","last_change":"2024-04-26T21:08:38.195331+0000","last_active":"2024-04-26T21:08:39.211320+0000","last_peered":"2024-04-26T21:08:39.211320+0000","last_clean":"2024-04-26T21:08:39.211320+0000","last_became_active":"2024-04-26T21:08:38.195002+0000","last_became_peered":"2024-04-26T21:08:38.195002+0000","last_unstale":"2024-04-26T21:08:39.211320+0000","last_undegraded":"2024-04-26T21:08:39.211320+0000","last_fullsized":"2024-04-26T21:08:39.211320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:42.506444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217273+0000","last_change":"2024-04-26T21:08:37.172072+0000","last_active":"2024-04-26T21:08:39.217273+0000","last_peered":"2024-04-26T21:08:39.217273+0000","last_clean":"2024-04-26T21:08:39.217273+0000","last_became_active":"2024-04-26T21:08:37.171778+0000","last_became_peered":"2024-04-26T21:08:37.171778+0000","last_unstale":"2024-04-26T21:08:39.217273+0000","last_undegraded":"2024-04-26T21:08:39.217273+0000","last_fullsized":"2024-04-26T21:08:39.217273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:00.488332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194798+0000","last_change":"2024-04-26T21:08:38.202960+0000","last_active":"2024-04-26T21:08:39.194798+0000","last_peered":"2024-04-26T21:08:39.194798+0000","last_clean":"2024-04-26T21:08:39.194798+0000","last_became_active":"2024-04-26T21:08:38.202642+0000","last_became_peered":"2024-04-26T21:08:38.202642+0000","last_unstale":"2024-04-26T21:08:39.194798+0000","last_undegraded":"2024-04-26T21:08:39.194798+0000","last_fullsized":"2024-04-26T21:08:39.194798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:55.797399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194905+0000","last_change":"2024-04-26T21:08:37.174576+0000","last_active":"2024-04-26T21:08:39.194905+0000","last_peered":"2024-04-26T21:08:39.194905+0000","last_clean":"2024-04-26T21:08:39.194905+0000","last_became_active":"2024-04-26T21:08:37.174223+0000","last_became_peered":"2024-04-26T21:08:37.174223+0000","last_unstale":"2024-04-26T21:08:39.194905+0000","last_undegraded":"2024-04-26T21:08:39.194905+0000","last_fullsized":"2024-04-26T21:08:39.194905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:00.488332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712374+0000","last_change":"2024-04-26T21:08:38.210232+0000","last_active":"2024-04-26T21:08:39.712374+0000","last_peered":"2024-04-26T21:08:39.712374+0000","last_clean":"2024-04-26T21:08:39.712374+0000","last_became_active":"2024-04-26T21:08:38.207957+0000","last_became_peered":"2024-04-26T21:08:38.207957+0000","last_unstale":"2024-04-26T21:08:39.712374+0000","last_undegraded":"2024-04-26T21:08:39.712374+0000","last_fullsized":"2024-04-26T21:08:39.712374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:29.659217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211019+0000","last_change":"2024-04-26T21:08:37.166766+0000","last_active":"2024-04-26T21:08:39.211019+0000","last_peered":"2024-04-26T21:08:39.211019+0000","last_clean":"2024-04-26T21:08:39.211019+0000","last_became_active":"2024-04-26T21:08:37.166441+0000","last_became_peered":"2024-04-26T21:08:37.166441+0000","last_unstale":"2024-04-26T21:08:39.211019+0000","last_undegraded":"2024-04-26T21:08:39.211019+0000","last_fullsized":"2024-04-26T21:08:39.211019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:58.240486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195784+0000","last_change":"2024-04-26T21:08:38.209109+0000","last_active":"2024-04-26T21:08:39.195784+0000","last_peered":"2024-04-26T21:08:39.195784+0000","last_clean":"2024-04-26T21:08:39.195784+0000","last_became_active":"2024-04-26T21:08:38.208779+0000","last_became_peered":"2024-04-26T21:08:38.208779+0000","last_unstale":"2024-04-26T21:08:39.195784+0000","last_undegraded":"2024-04-26T21:08:39.195784+0000","last_fullsized":"2024-04-26T21:08:39.195784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:59.255528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217053+0000","last_change":"2024-04-26T21:08:37.176062+0000","last_active":"2024-04-26T21:08:39.217053+0000","last_peered":"2024-04-26T21:08:39.217053+0000","last_clean":"2024-04-26T21:08:39.217053+0000","last_became_active":"2024-04-26T21:08:37.175757+0000","last_became_peered":"2024-04-26T21:08:37.175757+0000","last_unstale":"2024-04-26T21:08:39.217053+0000","last_undegraded":"2024-04-26T21:08:39.217053+0000","last_fullsized":"2024-04-26T21:08:39.217053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:58.240486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711341+0000","last_change":"2024-04-26T21:08:38.199662+0000","last_active":"2024-04-26T21:08:39.711341+0000","last_peered":"2024-04-26T21:08:39.711341+0000","last_clean":"2024-04-26T21:08:39.711341+0000","last_became_active":"2024-04-26T21:08:38.198909+0000","last_became_peered":"2024-04-26T21:08:38.198909+0000","last_unstale":"2024-04-26T21:08:39.711341+0000","last_undegraded":"2024-04-26T21:08:39.711341+0000","last_fullsized":"2024-04-26T21:08:39.711341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:46.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.713546+0000","last_change":"2024-04-26T21:08:37.149949+0000","last_active":"2024-04-26T21:08:39.713546+0000","last_peered":"2024-04-26T21:08:39.713546+0000","last_clean":"2024-04-26T21:08:39.713546+0000","last_became_active":"2024-04-26T21:08:37.149579+0000","last_became_peered":"2024-04-26T21:08:37.149579+0000","last_unstale":"2024-04-26T21:08:39.713546+0000","last_undegraded":"2024-04-26T21:08:39.713546+0000","last_fullsized":"2024-04-26T21:08:39.713546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:29.045350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712396+0000","last_change":"2024-04-26T21:08:38.196010+0000","last_active":"2024-04-26T21:08:39.712396+0000","last_peered":"2024-04-26T21:08:39.712396+0000","last_clean":"2024-04-26T21:08:39.712396+0000","last_became_active":"2024-04-26T21:08:38.195650+0000","last_became_peered":"2024-04-26T21:08:38.195650+0000","last_unstale":"2024-04-26T21:08:39.712396+0000","last_undegraded":"2024-04-26T21:08:39.712396+0000","last_fullsized":"2024-04-26T21:08:39.712396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:59.807950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194744+0000","last_change":"2024-04-26T21:08:37.170877+0000","last_active":"2024-04-26T21:08:39.194744+0000","last_peered":"2024-04-26T21:08:39.194744+0000","last_clean":"2024-04-26T21:08:39.194744+0000","last_became_active":"2024-04-26T21:08:37.170548+0000","last_became_peered":"2024-04-26T21:08:37.170548+0000","last_unstale":"2024-04-26T21:08:39.194744+0000","last_undegraded":"2024-04-26T21:08:39.194744+0000","last_fullsized":"2024-04-26T21:08:39.194744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:58.240486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711189+0000","last_change":"2024-04-26T21:08:38.217551+0000","last_active":"2024-04-26T21:08:39.711189+0000","last_peered":"2024-04-26T21:08:39.711189+0000","last_clean":"2024-04-26T21:08:39.711189+0000","last_became_active":"2024-04-26T21:08:38.217228+0000","last_became_peered":"2024-04-26T21:08:38.217228+0000","last_unstale":"2024-04-26T21:08:39.711189+0000","last_undegraded":"2024-04-26T21:08:39.711189+0000","last_fullsized":"2024-04-26T21:08:39.711189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:00.305226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.216714+0000","last_change":"2024-04-26T21:08:37.170243+0000","last_active":"2024-04-26T21:08:39.216714+0000","last_peered":"2024-04-26T21:08:39.216714+0000","last_clean":"2024-04-26T21:08:39.216714+0000","last_became_active":"2024-04-26T21:08:37.169995+0000","last_became_peered":"2024-04-26T21:08:37.169995+0000","last_unstale":"2024-04-26T21:08:39.216714+0000","last_undegraded":"2024-04-26T21:08:39.216714+0000","last_fullsized":"2024-04-26T21:08:39.216714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:18.839848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.216725+0000","last_change":"2024-04-26T21:08:38.200818+0000","last_active":"2024-04-26T21:08:39.216725+0000","last_peered":"2024-04-26T21:08:39.216725+0000","last_clean":"2024-04-26T21:08:39.216725+0000","last_became_active":"2024-04-26T21:08:38.198256+0000","last_became_peered":"2024-04-26T21:08:38.198256+0000","last_unstale":"2024-04-26T21:08:39.216725+0000","last_undegraded":"2024-04-26T21:08:39.216725+0000","last_fullsized":"2024-04-26T21:08:39.216725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:01.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.191066+0000","last_change":"2024-04-26T21:08:37.170807+0000","last_active":"2024-04-26T21:08:39.191066+0000","last_peered":"2024-04-26T21:08:39.191066+0000","last_clean":"2024-04-26T21:08:39.191066+0000","last_became_active":"2024-04-26T21:08:37.170428+0000","last_became_peered":"2024-04-26T21:08:37.170428+0000","last_unstale":"2024-04-26T21:08:39.191066+0000","last_undegraded":"2024-04-26T21:08:39.191066+0000","last_fullsized":"2024-04-26T21:08:39.191066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:34.683220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711663+0000","last_change":"2024-04-26T21:08:38.209564+0000","last_active":"2024-04-26T21:08:39.711663+0000","last_peered":"2024-04-26T21:08:39.711663+0000","last_clean":"2024-04-26T21:08:39.711663+0000","last_became_active":"2024-04-26T21:08:38.209194+0000","last_became_peered":"2024-04-26T21:08:38.209194+0000","last_unstale":"2024-04-26T21:08:39.711663+0000","last_undegraded":"2024-04-26T21:08:39.711663+0000","last_fullsized":"2024-04-26T21:08:39.711663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:35.132498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.196061+0000","last_change":"2024-04-26T21:08:37.156599+0000","last_active":"2024-04-26T21:08:39.196061+0000","last_peered":"2024-04-26T21:08:39.196061+0000","last_clean":"2024-04-26T21:08:39.196061+0000","last_became_active":"2024-04-26T21:08:37.154195+0000","last_became_peered":"2024-04-26T21:08:37.154195+0000","last_unstale":"2024-04-26T21:08:39.196061+0000","last_undegraded":"2024-04-26T21:08:39.196061+0000","last_fullsized":"2024-04-26T21:08:39.196061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:00.067702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711404+0000","last_change":"2024-04-26T21:08:38.209445+0000","last_active":"2024-04-26T21:08:39.711404+0000","last_peered":"2024-04-26T21:08:39.711404+0000","last_clean":"2024-04-26T21:08:39.711404+0000","last_became_active":"2024-04-26T21:08:38.209134+0000","last_became_peered":"2024-04-26T21:08:38.209134+0000","last_unstale":"2024-04-26T21:08:39.711404+0000","last_undegraded":"2024-04-26T21:08:39.711404+0000","last_fullsized":"2024-04-26T21:08:39.711404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:44.324217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.191281+0000","last_change":"2024-04-26T21:08:37.176762+0000","last_active":"2024-04-26T21:08:39.191281+0000","last_peered":"2024-04-26T21:08:39.191281+0000","last_clean":"2024-04-26T21:08:39.191281+0000","last_became_active":"2024-04-26T21:08:37.176369+0000","last_became_peered":"2024-04-26T21:08:37.176369+0000","last_unstale":"2024-04-26T21:08:39.191281+0000","last_undegraded":"2024-04-26T21:08:39.191281+0000","last_fullsized":"2024-04-26T21:08:39.191281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:00.488332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712217+0000","last_change":"2024-04-26T21:08:38.211784+0000","last_active":"2024-04-26T21:08:39.712217+0000","last_peered":"2024-04-26T21:08:39.712217+0000","last_clean":"2024-04-26T21:08:39.712217+0000","last_became_active":"2024-04-26T21:08:38.210121+0000","last_became_peered":"2024-04-26T21:08:38.210121+0000","last_unstale":"2024-04-26T21:08:39.712217+0000","last_undegraded":"2024-04-26T21:08:39.712217+0000","last_fullsized":"2024-04-26T21:08:39.712217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:46.194523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211130+0000","last_change":"2024-04-26T21:08:37.167350+0000","last_active":"2024-04-26T21:08:39.211130+0000","last_peered":"2024-04-26T21:08:39.211130+0000","last_clean":"2024-04-26T21:08:39.211130+0000","last_became_active":"2024-04-26T21:08:37.166952+0000","last_became_peered":"2024-04-26T21:08:37.166952+0000","last_unstale":"2024-04-26T21:08:39.211130+0000","last_undegraded":"2024-04-26T21:08:39.211130+0000","last_fullsized":"2024-04-26T21:08:39.211130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:30.628461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195797+0000","last_change":"2024-04-26T21:08:38.202682+0000","last_active":"2024-04-26T21:08:39.195797+0000","last_peered":"2024-04-26T21:08:39.195797+0000","last_clean":"2024-04-26T21:08:39.195797+0000","last_became_active":"2024-04-26T21:08:38.202299+0000","last_became_peered":"2024-04-26T21:08:38.202299+0000","last_unstale":"2024-04-26T21:08:39.195797+0000","last_undegraded":"2024-04-26T21:08:39.195797+0000","last_fullsized":"2024-04-26T21:08:39.195797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:19.756896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.191430+0000","last_change":"2024-04-26T21:08:37.168687+0000","last_active":"2024-04-26T21:08:39.191430+0000","last_peered":"2024-04-26T21:08:39.191430+0000","last_clean":"2024-04-26T21:08:39.191430+0000","last_became_active":"2024-04-26T21:08:37.168302+0000","last_became_peered":"2024-04-26T21:08:37.168302+0000","last_unstale":"2024-04-26T21:08:39.191430+0000","last_undegraded":"2024-04-26T21:08:39.191430+0000","last_fullsized":"2024-04-26T21:08:39.191430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:48:02.545851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711826+0000","last_change":"2024-04-26T21:08:38.208610+0000","last_active":"2024-04-26T21:08:39.711826+0000","last_peered":"2024-04-26T21:08:39.711826+0000","last_clean":"2024-04-26T21:08:39.711826+0000","last_became_active":"2024-04-26T21:08:38.206735+0000","last_became_peered":"2024-04-26T21:08:38.206735+0000","last_unstale":"2024-04-26T21:08:39.711826+0000","last_undegraded":"2024-04-26T21:08:39.711826+0000","last_fullsized":"2024-04-26T21:08:39.711826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:54.952928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.710746+0000","last_change":"2024-04-26T21:08:33.504911+0000","last_active":"2024-04-26T21:08:39.710746+0000","last_peered":"2024-04-26T21:08:39.710746+0000","last_clean":"2024-04-26T21:08:39.710746+0000","last_became_active":"2024-04-26T21:08:33.503667+0000","last_became_peered":"2024-04-26T21:08:33.503667+0000","last_unstale":"2024-04-26T21:08:39.710746+0000","last_undegraded":"2024-04-26T21:08:39.710746+0000","last_fullsized":"2024-04-26T21:08:39.710746+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:32.013907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:32.013907+0000","last_clean_scrub_stamp":"2024-04-26T21:08:32.013907+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:09.596153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194166+0000","last_change":"2024-04-26T21:08:37.161497+0000","last_active":"2024-04-26T21:08:39.194166+0000","last_peered":"2024-04-26T21:08:39.194166+0000","last_clean":"2024-04-26T21:08:39.194166+0000","last_became_active":"2024-04-26T21:08:37.161144+0000","last_became_peered":"2024-04-26T21:08:37.161144+0000","last_unstale":"2024-04-26T21:08:39.194166+0000","last_undegraded":"2024-04-26T21:08:39.194166+0000","last_fullsized":"2024-04-26T21:08:39.194166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:18.839848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712097+0000","last_change":"2024-04-26T21:08:38.211227+0000","last_active":"2024-04-26T21:08:39.712097+0000","last_peered":"2024-04-26T21:08:39.712097+0000","last_clean":"2024-04-26T21:08:39.712097+0000","last_became_active":"2024-04-26T21:08:38.210882+0000","last_became_peered":"2024-04-26T21:08:38.210882+0000","last_unstale":"2024-04-26T21:08:39.712097+0000","last_undegraded":"2024-04-26T21:08:39.712097+0000","last_fullsized":"2024-04-26T21:08:39.712097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:42.506444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.710561+0000","last_change":"2024-04-26T21:08:37.177216+0000","last_active":"2024-04-26T21:08:39.710561+0000","last_peered":"2024-04-26T21:08:39.710561+0000","last_clean":"2024-04-26T21:08:39.710561+0000","last_became_active":"2024-04-26T21:08:37.176919+0000","last_became_peered":"2024-04-26T21:08:37.176919+0000","last_unstale":"2024-04-26T21:08:39.710561+0000","last_undegraded":"2024-04-26T21:08:39.710561+0000","last_fullsized":"2024-04-26T21:08:39.710561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:58.240486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712236+0000","last_change":"2024-04-26T21:08:38.191994+0000","last_active":"2024-04-26T21:08:39.712236+0000","last_peered":"2024-04-26T21:08:39.712236+0000","last_clean":"2024-04-26T21:08:39.712236+0000","last_became_active":"2024-04-26T21:08:38.191567+0000","last_became_peered":"2024-04-26T21:08:38.191567+0000","last_unstale":"2024-04-26T21:08:39.712236+0000","last_undegraded":"2024-04-26T21:08:39.712236+0000","last_fullsized":"2024-04-26T21:08:39.712236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:01.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.191022+0000","last_change":"2024-04-26T21:08:37.172777+0000","last_active":"2024-04-26T21:08:39.191022+0000","last_peered":"2024-04-26T21:08:39.191022+0000","last_clean":"2024-04-26T21:08:39.191022+0000","last_became_active":"2024-04-26T21:08:37.171865+0000","last_became_peered":"2024-04-26T21:08:37.171865+0000","last_unstale":"2024-04-26T21:08:39.191022+0000","last_undegraded":"2024-04-26T21:08:39.191022+0000","last_fullsized":"2024-04-26T21:08:39.191022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:18.839848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.210976+0000","last_change":"2024-04-26T21:08:38.200547+0000","last_active":"2024-04-26T21:08:39.210976+0000","last_peered":"2024-04-26T21:08:39.210976+0000","last_clean":"2024-04-26T21:08:39.210976+0000","last_became_active":"2024-04-26T21:08:38.200025+0000","last_became_peered":"2024-04-26T21:08:38.200025+0000","last_unstale":"2024-04-26T21:08:39.210976+0000","last_undegraded":"2024-04-26T21:08:39.210976+0000","last_fullsized":"2024-04-26T21:08:39.210976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:59.807950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194881+0000","last_change":"2024-04-26T21:08:37.170761+0000","last_active":"2024-04-26T21:08:39.194881+0000","last_peered":"2024-04-26T21:08:39.194881+0000","last_clean":"2024-04-26T21:08:39.194881+0000","last_became_active":"2024-04-26T21:08:37.170431+0000","last_became_peered":"2024-04-26T21:08:37.170431+0000","last_unstale":"2024-04-26T21:08:39.194881+0000","last_undegraded":"2024-04-26T21:08:39.194881+0000","last_fullsized":"2024-04-26T21:08:39.194881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:59.977930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195387+0000","last_change":"2024-04-26T21:08:38.213352+0000","last_active":"2024-04-26T21:08:39.195387+0000","last_peered":"2024-04-26T21:08:39.195387+0000","last_clean":"2024-04-26T21:08:39.195387+0000","last_became_active":"2024-04-26T21:08:38.201057+0000","last_became_peered":"2024-04-26T21:08:38.201057+0000","last_unstale":"2024-04-26T21:08:39.195387+0000","last_undegraded":"2024-04-26T21:08:39.195387+0000","last_fullsized":"2024-04-26T21:08:39.195387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:35.698263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.210868+0000","last_change":"2024-04-26T21:08:37.169249+0000","last_active":"2024-04-26T21:08:39.210868+0000","last_peered":"2024-04-26T21:08:39.210868+0000","last_clean":"2024-04-26T21:08:39.210868+0000","last_became_active":"2024-04-26T21:08:37.168910+0000","last_became_peered":"2024-04-26T21:08:37.168910+0000","last_unstale":"2024-04-26T21:08:39.210868+0000","last_undegraded":"2024-04-26T21:08:39.210868+0000","last_fullsized":"2024-04-26T21:08:39.210868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:34.683220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194515+0000","last_change":"2024-04-26T21:08:38.206722+0000","last_active":"2024-04-26T21:08:39.194515+0000","last_peered":"2024-04-26T21:08:39.194515+0000","last_clean":"2024-04-26T21:08:39.194515+0000","last_became_active":"2024-04-26T21:08:38.205394+0000","last_became_peered":"2024-04-26T21:08:38.205394+0000","last_unstale":"2024-04-26T21:08:39.194515+0000","last_undegraded":"2024-04-26T21:08:39.194515+0000","last_fullsized":"2024-04-26T21:08:39.194515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:44.324217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712111+0000","last_change":"2024-04-26T21:08:37.178417+0000","last_active":"2024-04-26T21:08:39.712111+0000","last_peered":"2024-04-26T21:08:39.712111+0000","last_clean":"2024-04-26T21:08:39.712111+0000","last_became_active":"2024-04-26T21:08:37.178031+0000","last_became_peered":"2024-04-26T21:08:37.178031+0000","last_unstale":"2024-04-26T21:08:39.712111+0000","last_undegraded":"2024-04-26T21:08:39.712111+0000","last_fullsized":"2024-04-26T21:08:39.712111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:54.872853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.210824+0000","last_change":"2024-04-26T21:08:38.200358+0000","last_active":"2024-04-26T21:08:39.210824+0000","last_peered":"2024-04-26T21:08:39.210824+0000","last_clean":"2024-04-26T21:08:39.210824+0000","last_became_active":"2024-04-26T21:08:38.200001+0000","last_became_peered":"2024-04-26T21:08:38.200001+0000","last_unstale":"2024-04-26T21:08:39.210824+0000","last_undegraded":"2024-04-26T21:08:39.210824+0000","last_fullsized":"2024-04-26T21:08:39.210824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:01.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.215005+0000","last_change":"2024-04-26T21:08:37.178473+0000","last_active":"2024-04-26T21:08:39.215005+0000","last_peered":"2024-04-26T21:08:39.215005+0000","last_clean":"2024-04-26T21:08:39.215005+0000","last_became_active":"2024-04-26T21:08:37.178079+0000","last_became_peered":"2024-04-26T21:08:37.178079+0000","last_unstale":"2024-04-26T21:08:39.215005+0000","last_undegraded":"2024-04-26T21:08:39.215005+0000","last_fullsized":"2024-04-26T21:08:39.215005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:31.424853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712508+0000","last_change":"2024-04-26T21:08:38.212830+0000","last_active":"2024-04-26T21:08:39.712508+0000","last_peered":"2024-04-26T21:08:39.712508+0000","last_clean":"2024-04-26T21:08:39.712508+0000","last_became_active":"2024-04-26T21:08:38.212327+0000","last_became_peered":"2024-04-26T21:08:38.212327+0000","last_unstale":"2024-04-26T21:08:39.712508+0000","last_undegraded":"2024-04-26T21:08:39.712508+0000","last_fullsized":"2024-04-26T21:08:39.712508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:26.983477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217434+0000","last_change":"2024-04-26T21:08:37.172503+0000","last_active":"2024-04-26T21:08:39.217434+0000","last_peered":"2024-04-26T21:08:39.217434+0000","last_clean":"2024-04-26T21:08:39.217434+0000","last_became_active":"2024-04-26T21:08:37.172129+0000","last_became_peered":"2024-04-26T21:08:37.172129+0000","last_unstale":"2024-04-26T21:08:39.217434+0000","last_undegraded":"2024-04-26T21:08:39.217434+0000","last_fullsized":"2024-04-26T21:08:39.217434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:34.683220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712025+0000","last_change":"2024-04-26T21:08:38.192302+0000","last_active":"2024-04-26T21:08:39.712025+0000","last_peered":"2024-04-26T21:08:39.712025+0000","last_clean":"2024-04-26T21:08:39.712025+0000","last_became_active":"2024-04-26T21:08:38.191741+0000","last_became_peered":"2024-04-26T21:08:38.191741+0000","last_unstale":"2024-04-26T21:08:39.712025+0000","last_undegraded":"2024-04-26T21:08:39.712025+0000","last_fullsized":"2024-04-26T21:08:39.712025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:48.443740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.710597+0000","last_change":"2024-04-26T21:08:37.178123+0000","last_active":"2024-04-26T21:08:39.710597+0000","last_peered":"2024-04-26T21:08:39.710597+0000","last_clean":"2024-04-26T21:08:39.710597+0000","last_became_active":"2024-04-26T21:08:37.177831+0000","last_became_peered":"2024-04-26T21:08:37.177831+0000","last_unstale":"2024-04-26T21:08:39.710597+0000","last_undegraded":"2024-04-26T21:08:39.710597+0000","last_fullsized":"2024-04-26T21:08:39.710597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:59.977930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.196679+0000","last_change":"2024-04-26T21:08:38.212239+0000","last_active":"2024-04-26T21:08:39.196679+0000","last_peered":"2024-04-26T21:08:39.196679+0000","last_clean":"2024-04-26T21:08:39.196679+0000","last_became_active":"2024-04-26T21:08:38.209717+0000","last_became_peered":"2024-04-26T21:08:38.209717+0000","last_unstale":"2024-04-26T21:08:39.196679+0000","last_undegraded":"2024-04-26T21:08:39.196679+0000","last_fullsized":"2024-04-26T21:08:39.196679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:31.643503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211322+0000","last_change":"2024-04-26T21:08:37.173887+0000","last_active":"2024-04-26T21:08:39.211322+0000","last_peered":"2024-04-26T21:08:39.211322+0000","last_clean":"2024-04-26T21:08:39.211322+0000","last_became_active":"2024-04-26T21:08:37.173034+0000","last_became_peered":"2024-04-26T21:08:37.173034+0000","last_unstale":"2024-04-26T21:08:39.211322+0000","last_undegraded":"2024-04-26T21:08:39.211322+0000","last_fullsized":"2024-04-26T21:08:39.211322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:00.488332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712910+0000","last_change":"2024-04-26T21:08:38.197901+0000","last_active":"2024-04-26T21:08:39.712910+0000","last_peered":"2024-04-26T21:08:39.712910+0000","last_clean":"2024-04-26T21:08:39.712910+0000","last_became_active":"2024-04-26T21:08:38.197592+0000","last_became_peered":"2024-04-26T21:08:38.197592+0000","last_unstale":"2024-04-26T21:08:39.712910+0000","last_undegraded":"2024-04-26T21:08:39.712910+0000","last_fullsized":"2024-04-26T21:08:39.712910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:46.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211038+0000","last_change":"2024-04-26T21:08:37.175903+0000","last_active":"2024-04-26T21:08:39.211038+0000","last_peered":"2024-04-26T21:08:39.211038+0000","last_clean":"2024-04-26T21:08:39.211038+0000","last_became_active":"2024-04-26T21:08:37.175605+0000","last_became_peered":"2024-04-26T21:08:37.175605+0000","last_unstale":"2024-04-26T21:08:39.211038+0000","last_undegraded":"2024-04-26T21:08:39.211038+0000","last_fullsized":"2024-04-26T21:08:39.211038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:59.977930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711599+0000","last_change":"2024-04-26T21:08:38.208922+0000","last_active":"2024-04-26T21:08:39.711599+0000","last_peered":"2024-04-26T21:08:39.711599+0000","last_clean":"2024-04-26T21:08:39.711599+0000","last_became_active":"2024-04-26T21:08:38.208597+0000","last_became_peered":"2024-04-26T21:08:38.208597+0000","last_unstale":"2024-04-26T21:08:39.711599+0000","last_undegraded":"2024-04-26T21:08:39.711599+0000","last_fullsized":"2024-04-26T21:08:39.711599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:59.807950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711492+0000","last_change":"2024-04-26T21:08:37.173916+0000","last_active":"2024-04-26T21:08:39.711492+0000","last_peered":"2024-04-26T21:08:39.711492+0000","last_clean":"2024-04-26T21:08:39.711492+0000","last_became_active":"2024-04-26T21:08:37.173553+0000","last_became_peered":"2024-04-26T21:08:37.173553+0000","last_unstale":"2024-04-26T21:08:39.711492+0000","last_undegraded":"2024-04-26T21:08:39.711492+0000","last_fullsized":"2024-04-26T21:08:39.711492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:30.628461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217095+0000","last_change":"2024-04-26T21:08:38.212699+0000","last_active":"2024-04-26T21:08:39.217095+0000","last_peered":"2024-04-26T21:08:39.217095+0000","last_clean":"2024-04-26T21:08:39.217095+0000","last_became_active":"2024-04-26T21:08:38.212418+0000","last_became_peered":"2024-04-26T21:08:38.212418+0000","last_unstale":"2024-04-26T21:08:39.217095+0000","last_undegraded":"2024-04-26T21:08:39.217095+0000","last_fullsized":"2024-04-26T21:08:39.217095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:59.807950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194366+0000","last_change":"2024-04-26T21:08:38.211036+0000","last_active":"2024-04-26T21:08:39.194366+0000","last_peered":"2024-04-26T21:08:39.194366+0000","last_clean":"2024-04-26T21:08:39.194366+0000","last_became_active":"2024-04-26T21:08:38.207129+0000","last_became_peered":"2024-04-26T21:08:38.207129+0000","last_unstale":"2024-04-26T21:08:39.194366+0000","last_undegraded":"2024-04-26T21:08:39.194366+0000","last_fullsized":"2024-04-26T21:08:39.194366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:46.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711252+0000","last_change":"2024-04-26T21:08:37.179811+0000","last_active":"2024-04-26T21:08:39.711252+0000","last_peered":"2024-04-26T21:08:39.711252+0000","last_clean":"2024-04-26T21:08:39.711252+0000","last_became_active":"2024-04-26T21:08:37.179425+0000","last_became_peered":"2024-04-26T21:08:37.179425+0000","last_unstale":"2024-04-26T21:08:39.711252+0000","last_undegraded":"2024-04-26T21:08:39.711252+0000","last_fullsized":"2024-04-26T21:08:39.711252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:31.424853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195669+0000","last_change":"2024-04-26T21:08:38.207278+0000","last_active":"2024-04-26T21:08:39.195669+0000","last_peered":"2024-04-26T21:08:39.195669+0000","last_clean":"2024-04-26T21:08:39.195669+0000","last_became_active":"2024-04-26T21:08:38.204849+0000","last_became_peered":"2024-04-26T21:08:38.204849+0000","last_unstale":"2024-04-26T21:08:39.195669+0000","last_undegraded":"2024-04-26T21:08:39.195669+0000","last_fullsized":"2024-04-26T21:08:39.195669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:54.952928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711653+0000","last_change":"2024-04-26T21:08:37.175992+0000","last_active":"2024-04-26T21:08:39.711653+0000","last_peered":"2024-04-26T21:08:39.711653+0000","last_clean":"2024-04-26T21:08:39.711653+0000","last_became_active":"2024-04-26T21:08:37.175682+0000","last_became_peered":"2024-04-26T21:08:37.175682+0000","last_unstale":"2024-04-26T21:08:39.711653+0000","last_undegraded":"2024-04-26T21:08:39.711653+0000","last_fullsized":"2024-04-26T21:08:39.711653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:33.374984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.710839+0000","last_change":"2024-04-26T21:08:38.207957+0000","last_active":"2024-04-26T21:08:39.710839+0000","last_peered":"2024-04-26T21:08:39.710839+0000","last_clean":"2024-04-26T21:08:39.710839+0000","last_became_active":"2024-04-26T21:08:38.207573+0000","last_became_peered":"2024-04-26T21:08:38.207573+0000","last_unstale":"2024-04-26T21:08:39.710839+0000","last_undegraded":"2024-04-26T21:08:39.710839+0000","last_fullsized":"2024-04-26T21:08:39.710839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:02:14.307645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712093+0000","last_change":"2024-04-26T21:08:37.173718+0000","last_active":"2024-04-26T21:08:39.712093+0000","last_peered":"2024-04-26T21:08:39.712093+0000","last_clean":"2024-04-26T21:08:39.712093+0000","last_became_active":"2024-04-26T21:08:37.170846+0000","last_became_peered":"2024-04-26T21:08:37.170846+0000","last_unstale":"2024-04-26T21:08:39.712093+0000","last_undegraded":"2024-04-26T21:08:39.712093+0000","last_fullsized":"2024-04-26T21:08:39.712093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:48:02.545851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711803+0000","last_change":"2024-04-26T21:08:38.199029+0000","last_active":"2024-04-26T21:08:39.711803+0000","last_peered":"2024-04-26T21:08:39.711803+0000","last_clean":"2024-04-26T21:08:39.711803+0000","last_became_active":"2024-04-26T21:08:38.198720+0000","last_became_peered":"2024-04-26T21:08:38.198720+0000","last_unstale":"2024-04-26T21:08:39.711803+0000","last_undegraded":"2024-04-26T21:08:39.711803+0000","last_fullsized":"2024-04-26T21:08:39.711803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:40.618646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712817+0000","last_change":"2024-04-26T21:08:37.165057+0000","last_active":"2024-04-26T21:08:39.712817+0000","last_peered":"2024-04-26T21:08:39.712817+0000","last_clean":"2024-04-26T21:08:39.712817+0000","last_became_active":"2024-04-26T21:08:37.164781+0000","last_became_peered":"2024-04-26T21:08:37.164781+0000","last_unstale":"2024-04-26T21:08:39.712817+0000","last_undegraded":"2024-04-26T21:08:39.712817+0000","last_fullsized":"2024-04-26T21:08:39.712817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:17.499100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:08:48.248 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:08:48.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.494+0000 7fb716a00640 1 Processor -- start 2024-04-26T21:08:48.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.494+0000 7fb716a00640 1 -- start start 2024-04-26T21:08:48.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.495+0000 7fb716a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7100d1c80 0x7fb7100d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:48.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.495+0000 7fb716a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb710162ba0 0x7fb7100d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:48.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.495+0000 7fb716a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7100b61c0 con 0x7fb7100d2060 2024-04-26T21:08:48.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.495+0000 7fb716a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fb7100b5f70 con 0x7fb710162ba0 2024-04-26T21:08:48.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.495+0000 7fb716a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb7100d3830 con 0x7fb7100d1c80 2024-04-26T21:08:48.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.496+0000 7fb716000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb710162ba0 0x7fb7100d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:48.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.496+0000 7fb716000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb710162ba0 0x7fb7100d0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34954/0 (socket says 172.21.15.26:34954) 2024-04-26T21:08:48.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.496+0000 7fb716000640 1 -- 172.21.15.26:0/628787645 learned_addr learned my addr 172.21.15.26:0/628787645 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:48.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.496+0000 7fb715600640 1 --2- 172.21.15.26:0/628787645 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7100d1c80 0x7fb7100d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:48.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.496+0000 7fb716000640 1 -- 172.21.15.26:0/628787645 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7100d1c80 msgr2=0x7fb7100d3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:48.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.496+0000 7fb716000640 1 --2- 172.21.15.26:0/628787645 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7100d1c80 0x7fb7100d3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.496+0000 7fb716000640 1 -- 172.21.15.26:0/628787645 >> v1:172.21.15.26:6789/0 conn(0x7fb7100d2060 legacy=0x7fb710162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:48.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.496+0000 7fb716000640 1 -- 172.21.15.26:0/628787645 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7100b64a0 con 0x7fb710162ba0 2024-04-26T21:08:48.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.496+0000 7fb70fe00640 1 -- 172.21.15.26:0/628787645 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3318808755 0 0) 0x7fb7100b61c0 con 0x7fb7100d2060 2024-04-26T21:08:48.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.497+0000 7fb716000640 1 --2- 172.21.15.26:0/628787645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb710162ba0 0x7fb7100d0cf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb708009e50 tx=0x7fb70802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd64a97c81ebeca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:48.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.497+0000 7fb70fe00640 1 -- 172.21.15.26:0/628787645 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708036070 con 0x7fb710162ba0 2024-04-26T21:08:48.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.497+0000 7fb70fe00640 1 -- 172.21.15.26:0/628787645 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb708031600 con 0x7fb710162ba0 2024-04-26T21:08:48.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.497+0000 7fb70fe00640 1 -- 172.21.15.26:0/628787645 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7080319d0 con 0x7fb710162ba0 2024-04-26T21:08:48.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.498+0000 7fb716a00640 1 -- 172.21.15.26:0/628787645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb710162ba0 msgr2=0x7fb7100d0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:48.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.498+0000 7fb716a00640 1 --2- 172.21.15.26:0/628787645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb710162ba0 0x7fb7100d0cf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb708009e50 tx=0x7fb70802a680 comp rx=0 tx=0).stop 2024-04-26T21:08:48.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.498+0000 7fb716a00640 1 -- 172.21.15.26:0/628787645 shutdown_connections 2024-04-26T21:08:48.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.498+0000 7fb716a00640 1 --2- 172.21.15.26:0/628787645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb710162ba0 0x7fb7100d0cf0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.498+0000 7fb716a00640 1 --2- 172.21.15.26:0/628787645 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7100d1c80 0x7fb7100d3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.498+0000 7fb716a00640 1 -- 172.21.15.26:0/628787645 >> 172.21.15.26:0/628787645 conn(0x7fb7100d5020 msgr2=0x7fb7100d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:48.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.498+0000 7fb716a00640 1 -- 172.21.15.26:0/628787645 shutdown_connections 2024-04-26T21:08:48.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.498+0000 7fb716a00640 1 -- 172.21.15.26:0/628787645 wait complete. 2024-04-26T21:08:48.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.500+0000 7fb716a00640 1 Processor -- start 2024-04-26T21:08:48.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.500+0000 7fb716a00640 1 -- start start 2024-04-26T21:08:48.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.501+0000 7fb716a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7100d1c80 0x7fb71015c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:48.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.501+0000 7fb716a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7100d2060 0x7fb7100b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:48.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.501+0000 7fb716a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb710162ba0 0x7fb7100b9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:48.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.501+0000 7fb716a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb7100d5cc0 con 0x7fb710162ba0 2024-04-26T21:08:48.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.501+0000 7fb716a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb7100d5b40 con 0x7fb7100d1c80 2024-04-26T21:08:48.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.501+0000 7fb716a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb7100c8b70 con 0x7fb7100d2060 2024-04-26T21:08:48.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.501+0000 7fb716000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb710162ba0 0x7fb7100b9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:48.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.501+0000 7fb716000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb710162ba0 0x7fb7100b9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34962/0 (socket says 172.21.15.26:34962) 2024-04-26T21:08:48.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.501+0000 7fb716000640 1 -- 172.21.15.26:0/1200119611 learned_addr learned my addr 172.21.15.26:0/1200119611 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:48.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.501+0000 7fb716000640 1 -- 172.21.15.26:0/1200119611 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7100d2060 msgr2=0x7fb7100b8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:48.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.501+0000 7fb716000640 1 --2- 172.21.15.26:0/1200119611 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7100d2060 0x7fb7100b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.501+0000 7fb716000640 1 -- 172.21.15.26:0/1200119611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7100d1c80 msgr2=0x7fb71015c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:48.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.501+0000 7fb716000640 1 --2- 172.21.15.26:0/1200119611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7100d1c80 0x7fb71015c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.501+0000 7fb716000640 1 -- 172.21.15.26:0/1200119611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7101f5f90 con 0x7fb710162ba0 2024-04-26T21:08:48.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.502+0000 7fb716000640 1 --2- 172.21.15.26:0/1200119611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb710162ba0 0x7fb7100b9ee0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb708034040 tx=0x7fb708004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:48.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.502+0000 7fb70d600640 1 -- 172.21.15.26:0/1200119611 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708041020 con 0x7fb710162ba0 2024-04-26T21:08:48.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.502+0000 7fb70d600640 1 -- 172.21.15.26:0/1200119611 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb708007780 con 0x7fb710162ba0 2024-04-26T21:08:48.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.502+0000 7fb70d600640 1 -- 172.21.15.26:0/1200119611 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708036040 con 0x7fb710162ba0 2024-04-26T21:08:48.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.502+0000 7fb716a00640 1 -- 172.21.15.26:0/1200119611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7101f6230 con 0x7fb710162ba0 2024-04-26T21:08:48.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.502+0000 7fb716a00640 1 -- 172.21.15.26:0/1200119611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7101f67f0 con 0x7fb710162ba0 2024-04-26T21:08:48.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.503+0000 7fb716a00640 1 -- 172.21.15.26:0/1200119611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb71006a700 con 0x7fb710162ba0 2024-04-26T21:08:48.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.504+0000 7fb70d600640 1 -- 172.21.15.26:0/1200119611 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb708044020 con 0x7fb710162ba0 2024-04-26T21:08:48.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.506+0000 7fb70d600640 1 --2- 172.21.15.26:0/1200119611 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb6f006c920 0x7fb6f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:48.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.506+0000 7fb70d600640 1 -- 172.21.15.26:0/1200119611 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb7080ac8f0 con 0x7fb710162ba0 2024-04-26T21:08:48.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.506+0000 7fb715600640 1 --2- 172.21.15.26:0/1200119611 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb6f006c920 0x7fb6f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:48.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.507+0000 7fb715600640 1 --2- 172.21.15.26:0/1200119611 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb6f006c920 0x7fb6f006ede0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb704006fd0 tx=0x7fb704008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:48.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.510+0000 7fb70d600640 1 -- 172.21.15.26:0/1200119611 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb70804b030 con 0x7fb710162ba0 2024-04-26T21:08:48.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.706+0000 7fb716a00640 1 -- 172.21.15.26:0/1200119611 --> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb71006e2a0 con 0x7fb6f006c920 2024-04-26T21:08:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.717+0000 7fb70d600640 1 -- 172.21.15.26:0/1200119611 <== mgr.4104 v2:172.21.15.26:6832/3130430358 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7fb71006e2a0 con 0x7fb6f006c920 2024-04-26T21:08:48.722 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:48.723 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-26T21:08:48.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.723+0000 7fb716a00640 1 -- 172.21.15.26:0/1200119611 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb6f006c920 msgr2=0x7fb6f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:48.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.723+0000 7fb716a00640 1 --2- 172.21.15.26:0/1200119611 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb6f006c920 0x7fb6f006ede0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb704006fd0 tx=0x7fb704008040 comp rx=0 tx=0).stop 2024-04-26T21:08:48.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.723+0000 7fb716a00640 1 -- 172.21.15.26:0/1200119611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb710162ba0 msgr2=0x7fb7100b9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:48.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.723+0000 7fb716a00640 1 --2- 172.21.15.26:0/1200119611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb710162ba0 0x7fb7100b9ee0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb708034040 tx=0x7fb708004a40 comp rx=0 tx=0).stop 2024-04-26T21:08:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.723+0000 7fb716a00640 1 -- 172.21.15.26:0/1200119611 shutdown_connections 2024-04-26T21:08:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.723+0000 7fb716a00640 1 --2- 172.21.15.26:0/1200119611 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb6f006c920 0x7fb6f006ede0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.723+0000 7fb716a00640 1 --2- 172.21.15.26:0/1200119611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb710162ba0 0x7fb7100b9ee0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.723+0000 7fb716a00640 1 --2- 172.21.15.26:0/1200119611 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7100d2060 0x7fb7100b8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.723+0000 7fb716a00640 1 --2- 172.21.15.26:0/1200119611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7100d1c80 0x7fb71015c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.723+0000 7fb716a00640 1 -- 172.21.15.26:0/1200119611 >> 172.21.15.26:0/1200119611 conn(0x7fb7100d5020 msgr2=0x7fb7100caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.723+0000 7fb716a00640 1 -- 172.21.15.26:0/1200119611 shutdown_connections 2024-04-26T21:08:48.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.723+0000 7fb716a00640 1 -- 172.21.15.26:0/1200119611 wait complete. 2024-04-26T21:08:48.741 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-26T21:08:47.292515+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235764,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738956,"statfs":{"total":773094113280,"available":772852690944,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.131669"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711962+0000","last_change":"2024-04-26T21:08:38.198898+0000","last_active":"2024-04-26T21:08:39.711962+0000","last_peered":"2024-04-26T21:08:39.711962+0000","last_clean":"2024-04-26T21:08:39.711962+0000","last_became_active":"2024-04-26T21:08:38.198596+0000","last_became_peered":"2024-04-26T21:08:38.198596+0000","last_unstale":"2024-04-26T21:08:39.711962+0000","last_undegraded":"2024-04-26T21:08:39.711962+0000","last_fullsized":"2024-04-26T21:08:39.711962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:19.756896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211160+0000","last_change":"2024-04-26T21:08:38.194080+0000","last_active":"2024-04-26T21:08:39.211160+0000","last_peered":"2024-04-26T21:08:39.211160+0000","last_clean":"2024-04-26T21:08:39.211160+0000","last_became_active":"2024-04-26T21:08:38.193534+0000","last_became_peered":"2024-04-26T21:08:38.193534+0000","last_unstale":"2024-04-26T21:08:39.211160+0000","last_undegraded":"2024-04-26T21:08:39.211160+0000","last_fullsized":"2024-04-26T21:08:39.211160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:22.132136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712041+0000","last_change":"2024-04-26T21:08:38.198384+0000","last_active":"2024-04-26T21:08:39.712041+0000","last_peered":"2024-04-26T21:08:39.712041+0000","last_clean":"2024-04-26T21:08:39.712041+0000","last_became_active":"2024-04-26T21:08:38.198055+0000","last_became_peered":"2024-04-26T21:08:38.198055+0000","last_unstale":"2024-04-26T21:08:39.712041+0000","last_undegraded":"2024-04-26T21:08:39.712041+0000","last_fullsized":"2024-04-26T21:08:39.712041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:55.566474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194277+0000","last_change":"2024-04-26T21:08:38.205534+0000","last_active":"2024-04-26T21:08:39.194277+0000","last_peered":"2024-04-26T21:08:39.194277+0000","last_clean":"2024-04-26T21:08:39.194277+0000","last_became_active":"2024-04-26T21:08:38.205235+0000","last_became_peered":"2024-04-26T21:08:38.205235+0000","last_unstale":"2024-04-26T21:08:39.194277+0000","last_undegraded":"2024-04-26T21:08:39.194277+0000","last_fullsized":"2024-04-26T21:08:39.194277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:26.983477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711420+0000","last_change":"2024-04-26T21:08:38.200597+0000","last_active":"2024-04-26T21:08:39.711420+0000","last_peered":"2024-04-26T21:08:39.711420+0000","last_clean":"2024-04-26T21:08:39.711420+0000","last_became_active":"2024-04-26T21:08:38.197772+0000","last_became_peered":"2024-04-26T21:08:38.197772+0000","last_unstale":"2024-04-26T21:08:39.711420+0000","last_undegraded":"2024-04-26T21:08:39.711420+0000","last_fullsized":"2024-04-26T21:08:39.711420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:17.370683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.196006+0000","last_change":"2024-04-26T21:08:38.210251+0000","last_active":"2024-04-26T21:08:39.196006+0000","last_peered":"2024-04-26T21:08:39.196006+0000","last_clean":"2024-04-26T21:08:39.196006+0000","last_became_active":"2024-04-26T21:08:38.209628+0000","last_became_peered":"2024-04-26T21:08:38.209628+0000","last_unstale":"2024-04-26T21:08:39.196006+0000","last_undegraded":"2024-04-26T21:08:39.196006+0000","last_fullsized":"2024-04-26T21:08:39.196006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:28.069603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711557+0000","last_change":"2024-04-26T21:08:38.199522+0000","last_active":"2024-04-26T21:08:39.711557+0000","last_peered":"2024-04-26T21:08:39.711557+0000","last_clean":"2024-04-26T21:08:39.711557+0000","last_became_active":"2024-04-26T21:08:38.198122+0000","last_became_peered":"2024-04-26T21:08:38.198122+0000","last_unstale":"2024-04-26T21:08:39.711557+0000","last_undegraded":"2024-04-26T21:08:39.711557+0000","last_fullsized":"2024-04-26T21:08:39.711557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:55.797399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711087+0000","last_change":"2024-04-26T21:08:38.207966+0000","last_active":"2024-04-26T21:08:39.711087+0000","last_peered":"2024-04-26T21:08:39.711087+0000","last_clean":"2024-04-26T21:08:39.711087+0000","last_became_active":"2024-04-26T21:08:38.207617+0000","last_became_peered":"2024-04-26T21:08:38.207617+0000","last_unstale":"2024-04-26T21:08:39.711087+0000","last_undegraded":"2024-04-26T21:08:39.711087+0000","last_fullsized":"2024-04-26T21:08:39.711087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:55.566474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712128+0000","last_change":"2024-04-26T21:08:38.204121+0000","last_active":"2024-04-26T21:08:39.712128+0000","last_peered":"2024-04-26T21:08:39.712128+0000","last_clean":"2024-04-26T21:08:39.712128+0000","last_became_active":"2024-04-26T21:08:38.203125+0000","last_became_peered":"2024-04-26T21:08:38.203125+0000","last_unstale":"2024-04-26T21:08:39.712128+0000","last_undegraded":"2024-04-26T21:08:39.712128+0000","last_fullsized":"2024-04-26T21:08:39.712128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:36.343803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217352+0000","last_change":"2024-04-26T21:08:38.204308+0000","last_active":"2024-04-26T21:08:39.217352+0000","last_peered":"2024-04-26T21:08:39.217352+0000","last_clean":"2024-04-26T21:08:39.217352+0000","last_became_active":"2024-04-26T21:08:38.203942+0000","last_became_peered":"2024-04-26T21:08:38.203942+0000","last_unstale":"2024-04-26T21:08:39.217352+0000","last_undegraded":"2024-04-26T21:08:39.217352+0000","last_fullsized":"2024-04-26T21:08:39.217352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:42.506444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217214+0000","last_change":"2024-04-26T21:08:38.201929+0000","last_active":"2024-04-26T21:08:39.217214+0000","last_peered":"2024-04-26T21:08:39.217214+0000","last_clean":"2024-04-26T21:08:39.217214+0000","last_became_active":"2024-04-26T21:08:38.201646+0000","last_became_peered":"2024-04-26T21:08:38.201646+0000","last_unstale":"2024-04-26T21:08:39.217214+0000","last_undegraded":"2024-04-26T21:08:39.217214+0000","last_fullsized":"2024-04-26T21:08:39.217214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:40.618646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195574+0000","last_change":"2024-04-26T21:08:38.207847+0000","last_active":"2024-04-26T21:08:39.195574+0000","last_peered":"2024-04-26T21:08:39.195574+0000","last_clean":"2024-04-26T21:08:39.195574+0000","last_became_active":"2024-04-26T21:08:38.204908+0000","last_became_peered":"2024-04-26T21:08:38.204908+0000","last_unstale":"2024-04-26T21:08:39.195574+0000","last_undegraded":"2024-04-26T21:08:39.195574+0000","last_fullsized":"2024-04-26T21:08:39.195574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:00.992973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711183+0000","last_change":"2024-04-26T21:08:38.203775+0000","last_active":"2024-04-26T21:08:39.711183+0000","last_peered":"2024-04-26T21:08:39.711183+0000","last_clean":"2024-04-26T21:08:39.711183+0000","last_became_active":"2024-04-26T21:08:38.203139+0000","last_became_peered":"2024-04-26T21:08:38.203139+0000","last_unstale":"2024-04-26T21:08:39.711183+0000","last_undegraded":"2024-04-26T21:08:39.711183+0000","last_fullsized":"2024-04-26T21:08:39.711183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:01.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712480+0000","last_change":"2024-04-26T21:08:38.195656+0000","last_active":"2024-04-26T21:08:39.712480+0000","last_peered":"2024-04-26T21:08:39.712480+0000","last_clean":"2024-04-26T21:08:39.712480+0000","last_became_active":"2024-04-26T21:08:38.195240+0000","last_became_peered":"2024-04-26T21:08:38.195240+0000","last_unstale":"2024-04-26T21:08:39.712480+0000","last_undegraded":"2024-04-26T21:08:39.712480+0000","last_fullsized":"2024-04-26T21:08:39.712480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:16.232014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211031+0000","last_change":"2024-04-26T21:08:38.186523+0000","last_active":"2024-04-26T21:08:39.211031+0000","last_peered":"2024-04-26T21:08:39.211031+0000","last_clean":"2024-04-26T21:08:39.211031+0000","last_became_active":"2024-04-26T21:08:38.186159+0000","last_became_peered":"2024-04-26T21:08:38.186159+0000","last_unstale":"2024-04-26T21:08:39.211031+0000","last_undegraded":"2024-04-26T21:08:39.211031+0000","last_fullsized":"2024-04-26T21:08:39.211031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:40.618646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194033+0000","last_change":"2024-04-26T21:08:38.212958+0000","last_active":"2024-04-26T21:08:39.194033+0000","last_peered":"2024-04-26T21:08:39.194033+0000","last_clean":"2024-04-26T21:08:39.194033+0000","last_became_active":"2024-04-26T21:08:38.212666+0000","last_became_peered":"2024-04-26T21:08:38.212666+0000","last_unstale":"2024-04-26T21:08:39.194033+0000","last_undegraded":"2024-04-26T21:08:39.194033+0000","last_fullsized":"2024-04-26T21:08:39.194033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:42.506444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195541+0000","last_change":"2024-04-26T21:08:38.214751+0000","last_active":"2024-04-26T21:08:39.195541+0000","last_peered":"2024-04-26T21:08:39.195541+0000","last_clean":"2024-04-26T21:08:39.195541+0000","last_became_active":"2024-04-26T21:08:38.212857+0000","last_became_peered":"2024-04-26T21:08:38.212857+0000","last_unstale":"2024-04-26T21:08:39.195541+0000","last_undegraded":"2024-04-26T21:08:39.195541+0000","last_fullsized":"2024-04-26T21:08:39.195541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:35.132498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.191204+0000","last_change":"2024-04-26T21:08:38.198823+0000","last_active":"2024-04-26T21:08:39.191204+0000","last_peered":"2024-04-26T21:08:39.191204+0000","last_clean":"2024-04-26T21:08:39.191204+0000","last_became_active":"2024-04-26T21:08:38.198521+0000","last_became_peered":"2024-04-26T21:08:38.198521+0000","last_unstale":"2024-04-26T21:08:39.191204+0000","last_undegraded":"2024-04-26T21:08:39.191204+0000","last_fullsized":"2024-04-26T21:08:39.191204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:34.390026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712075+0000","last_change":"2024-04-26T21:08:38.200413+0000","last_active":"2024-04-26T21:08:39.712075+0000","last_peered":"2024-04-26T21:08:39.712075+0000","last_clean":"2024-04-26T21:08:39.712075+0000","last_became_active":"2024-04-26T21:08:38.200023+0000","last_became_peered":"2024-04-26T21:08:38.200023+0000","last_unstale":"2024-04-26T21:08:39.712075+0000","last_undegraded":"2024-04-26T21:08:39.712075+0000","last_fullsized":"2024-04-26T21:08:39.712075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:28.069603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.216976+0000","last_change":"2024-04-26T21:08:38.211354+0000","last_active":"2024-04-26T21:08:39.216976+0000","last_peered":"2024-04-26T21:08:39.216976+0000","last_clean":"2024-04-26T21:08:39.216976+0000","last_became_active":"2024-04-26T21:08:38.211016+0000","last_became_peered":"2024-04-26T21:08:38.211016+0000","last_unstale":"2024-04-26T21:08:39.216976+0000","last_undegraded":"2024-04-26T21:08:39.216976+0000","last_fullsized":"2024-04-26T21:08:39.216976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:29.659217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712867+0000","last_change":"2024-04-26T21:08:38.201685+0000","last_active":"2024-04-26T21:08:39.712867+0000","last_peered":"2024-04-26T21:08:39.712867+0000","last_clean":"2024-04-26T21:08:39.712867+0000","last_became_active":"2024-04-26T21:08:38.200467+0000","last_became_peered":"2024-04-26T21:08:38.200467+0000","last_unstale":"2024-04-26T21:08:39.712867+0000","last_undegraded":"2024-04-26T21:08:39.712867+0000","last_fullsized":"2024-04-26T21:08:39.712867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:40.618646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194361+0000","last_change":"2024-04-26T21:08:38.204033+0000","last_active":"2024-04-26T21:08:39.194361+0000","last_peered":"2024-04-26T21:08:39.194361+0000","last_clean":"2024-04-26T21:08:39.194361+0000","last_became_active":"2024-04-26T21:08:38.202425+0000","last_became_peered":"2024-04-26T21:08:38.202425+0000","last_unstale":"2024-04-26T21:08:39.194361+0000","last_undegraded":"2024-04-26T21:08:39.194361+0000","last_fullsized":"2024-04-26T21:08:39.194361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:00.305226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711233+0000","last_change":"2024-04-26T21:08:38.216188+0000","last_active":"2024-04-26T21:08:39.711233+0000","last_peered":"2024-04-26T21:08:39.711233+0000","last_clean":"2024-04-26T21:08:39.711233+0000","last_became_active":"2024-04-26T21:08:38.215866+0000","last_became_peered":"2024-04-26T21:08:38.215866+0000","last_unstale":"2024-04-26T21:08:39.711233+0000","last_undegraded":"2024-04-26T21:08:39.711233+0000","last_fullsized":"2024-04-26T21:08:39.711233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:29.659217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.193946+0000","last_change":"2024-04-26T21:08:38.213795+0000","last_active":"2024-04-26T21:08:39.193946+0000","last_peered":"2024-04-26T21:08:39.193946+0000","last_clean":"2024-04-26T21:08:39.193946+0000","last_became_active":"2024-04-26T21:08:38.211952+0000","last_became_peered":"2024-04-26T21:08:38.211952+0000","last_unstale":"2024-04-26T21:08:39.193946+0000","last_undegraded":"2024-04-26T21:08:39.193946+0000","last_fullsized":"2024-04-26T21:08:39.193946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:22.132136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712055+0000","last_change":"2024-04-26T21:08:38.211961+0000","last_active":"2024-04-26T21:08:39.712055+0000","last_peered":"2024-04-26T21:08:39.712055+0000","last_clean":"2024-04-26T21:08:39.712055+0000","last_became_active":"2024-04-26T21:08:38.211590+0000","last_became_peered":"2024-04-26T21:08:38.211590+0000","last_unstale":"2024-04-26T21:08:39.712055+0000","last_undegraded":"2024-04-26T21:08:39.712055+0000","last_fullsized":"2024-04-26T21:08:39.712055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:22.132136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217302+0000","last_change":"2024-04-26T21:08:38.200054+0000","last_active":"2024-04-26T21:08:39.217302+0000","last_peered":"2024-04-26T21:08:39.217302+0000","last_clean":"2024-04-26T21:08:39.217302+0000","last_became_active":"2024-04-26T21:08:38.199738+0000","last_became_peered":"2024-04-26T21:08:38.199738+0000","last_unstale":"2024-04-26T21:08:39.217302+0000","last_undegraded":"2024-04-26T21:08:39.217302+0000","last_fullsized":"2024-04-26T21:08:39.217302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:22.132136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.210875+0000","last_change":"2024-04-26T21:08:38.195577+0000","last_active":"2024-04-26T21:08:39.210875+0000","last_peered":"2024-04-26T21:08:39.210875+0000","last_clean":"2024-04-26T21:08:39.210875+0000","last_became_active":"2024-04-26T21:08:38.195284+0000","last_became_peered":"2024-04-26T21:08:38.195284+0000","last_unstale":"2024-04-26T21:08:39.210875+0000","last_undegraded":"2024-04-26T21:08:39.210875+0000","last_fullsized":"2024-04-26T21:08:39.210875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:29.659217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.713361+0000","last_change":"2024-04-26T21:08:38.202932+0000","last_active":"2024-04-26T21:08:39.713361+0000","last_peered":"2024-04-26T21:08:39.713361+0000","last_clean":"2024-04-26T21:08:39.713361+0000","last_became_active":"2024-04-26T21:08:38.202614+0000","last_became_peered":"2024-04-26T21:08:38.202614+0000","last_unstale":"2024-04-26T21:08:39.713361+0000","last_undegraded":"2024-04-26T21:08:39.713361+0000","last_fullsized":"2024-04-26T21:08:39.713361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:44.324217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.713283+0000","last_change":"2024-04-26T21:08:38.198429+0000","last_active":"2024-04-26T21:08:39.713283+0000","last_peered":"2024-04-26T21:08:39.713283+0000","last_clean":"2024-04-26T21:08:39.713283+0000","last_became_active":"2024-04-26T21:08:38.198144+0000","last_became_peered":"2024-04-26T21:08:38.198144+0000","last_unstale":"2024-04-26T21:08:39.713283+0000","last_undegraded":"2024-04-26T21:08:39.713283+0000","last_fullsized":"2024-04-26T21:08:39.713283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:22.132136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194610+0000","last_change":"2024-04-26T21:08:38.205890+0000","last_active":"2024-04-26T21:08:39.194610+0000","last_peered":"2024-04-26T21:08:39.194610+0000","last_clean":"2024-04-26T21:08:39.194610+0000","last_became_active":"2024-04-26T21:08:38.205583+0000","last_became_peered":"2024-04-26T21:08:38.205583+0000","last_unstale":"2024-04-26T21:08:39.194610+0000","last_undegraded":"2024-04-26T21:08:39.194610+0000","last_fullsized":"2024-04-26T21:08:39.194610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:17.370683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.190845+0000","last_change":"2024-04-26T21:08:38.204162+0000","last_active":"2024-04-26T21:08:39.190845+0000","last_peered":"2024-04-26T21:08:39.190845+0000","last_clean":"2024-04-26T21:08:39.190845+0000","last_became_active":"2024-04-26T21:08:38.203855+0000","last_became_peered":"2024-04-26T21:08:38.203855+0000","last_unstale":"2024-04-26T21:08:39.190845+0000","last_undegraded":"2024-04-26T21:08:39.190845+0000","last_fullsized":"2024-04-26T21:08:39.190845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:31.643503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.216694+0000","last_change":"2024-04-26T21:08:38.207713+0000","last_active":"2024-04-26T21:08:39.216694+0000","last_peered":"2024-04-26T21:08:39.216694+0000","last_clean":"2024-04-26T21:08:39.216694+0000","last_became_active":"2024-04-26T21:08:38.207384+0000","last_became_peered":"2024-04-26T21:08:38.207384+0000","last_unstale":"2024-04-26T21:08:39.216694+0000","last_undegraded":"2024-04-26T21:08:39.216694+0000","last_fullsized":"2024-04-26T21:08:39.216694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:52:55.566474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.196940+0000","last_change":"2024-04-26T21:08:38.194999+0000","last_active":"2024-04-26T21:08:39.196940+0000","last_peered":"2024-04-26T21:08:39.196940+0000","last_clean":"2024-04-26T21:08:39.196940+0000","last_became_active":"2024-04-26T21:08:38.194647+0000","last_became_peered":"2024-04-26T21:08:38.194647+0000","last_unstale":"2024-04-26T21:08:39.196940+0000","last_undegraded":"2024-04-26T21:08:39.196940+0000","last_fullsized":"2024-04-26T21:08:39.196940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:34.390026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711906+0000","last_change":"2024-04-26T21:08:37.168719+0000","last_active":"2024-04-26T21:08:39.711906+0000","last_peered":"2024-04-26T21:08:39.711906+0000","last_clean":"2024-04-26T21:08:39.711906+0000","last_became_active":"2024-04-26T21:08:37.168257+0000","last_became_peered":"2024-04-26T21:08:37.168257+0000","last_unstale":"2024-04-26T21:08:39.711906+0000","last_undegraded":"2024-04-26T21:08:39.711906+0000","last_fullsized":"2024-04-26T21:08:39.711906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:18.839848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.713404+0000","last_change":"2024-04-26T21:08:38.190358+0000","last_active":"2024-04-26T21:08:39.713404+0000","last_peered":"2024-04-26T21:08:39.713404+0000","last_clean":"2024-04-26T21:08:39.713404+0000","last_became_active":"2024-04-26T21:08:38.189905+0000","last_became_peered":"2024-04-26T21:08:38.189905+0000","last_unstale":"2024-04-26T21:08:39.713404+0000","last_undegraded":"2024-04-26T21:08:39.713404+0000","last_fullsized":"2024-04-26T21:08:39.713404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:42.506444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.216776+0000","last_change":"2024-04-26T21:08:37.177780+0000","last_active":"2024-04-26T21:08:39.216776+0000","last_peered":"2024-04-26T21:08:39.216776+0000","last_clean":"2024-04-26T21:08:39.216776+0000","last_became_active":"2024-04-26T21:08:37.177488+0000","last_became_peered":"2024-04-26T21:08:37.177488+0000","last_unstale":"2024-04-26T21:08:39.216776+0000","last_undegraded":"2024-04-26T21:08:39.216776+0000","last_fullsized":"2024-04-26T21:08:39.216776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:59.977930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711590+0000","last_change":"2024-04-26T21:08:37.171155+0000","last_active":"2024-04-26T21:08:39.711590+0000","last_peered":"2024-04-26T21:08:39.711590+0000","last_clean":"2024-04-26T21:08:39.711590+0000","last_became_active":"2024-04-26T21:08:37.170816+0000","last_became_peered":"2024-04-26T21:08:37.170816+0000","last_unstale":"2024-04-26T21:08:39.711590+0000","last_undegraded":"2024-04-26T21:08:39.711590+0000","last_fullsized":"2024-04-26T21:08:39.711590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:17.499100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712682+0000","last_change":"2024-04-26T21:08:38.206720+0000","last_active":"2024-04-26T21:08:39.712682+0000","last_peered":"2024-04-26T21:08:39.712682+0000","last_clean":"2024-04-26T21:08:39.712682+0000","last_became_active":"2024-04-26T21:08:38.206401+0000","last_became_peered":"2024-04-26T21:08:38.206401+0000","last_unstale":"2024-04-26T21:08:39.712682+0000","last_undegraded":"2024-04-26T21:08:39.712682+0000","last_fullsized":"2024-04-26T21:08:39.712682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:00.305226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194170+0000","last_change":"2024-04-26T21:08:37.161426+0000","last_active":"2024-04-26T21:08:39.194170+0000","last_peered":"2024-04-26T21:08:39.194170+0000","last_clean":"2024-04-26T21:08:39.194170+0000","last_became_active":"2024-04-26T21:08:37.161119+0000","last_became_peered":"2024-04-26T21:08:37.161119+0000","last_unstale":"2024-04-26T21:08:39.194170+0000","last_undegraded":"2024-04-26T21:08:39.194170+0000","last_fullsized":"2024-04-26T21:08:39.194170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:34.683220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211320+0000","last_change":"2024-04-26T21:08:38.195331+0000","last_active":"2024-04-26T21:08:39.211320+0000","last_peered":"2024-04-26T21:08:39.211320+0000","last_clean":"2024-04-26T21:08:39.211320+0000","last_became_active":"2024-04-26T21:08:38.195002+0000","last_became_peered":"2024-04-26T21:08:38.195002+0000","last_unstale":"2024-04-26T21:08:39.211320+0000","last_undegraded":"2024-04-26T21:08:39.211320+0000","last_fullsized":"2024-04-26T21:08:39.211320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:42.506444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217273+0000","last_change":"2024-04-26T21:08:37.172072+0000","last_active":"2024-04-26T21:08:39.217273+0000","last_peered":"2024-04-26T21:08:39.217273+0000","last_clean":"2024-04-26T21:08:39.217273+0000","last_became_active":"2024-04-26T21:08:37.171778+0000","last_became_peered":"2024-04-26T21:08:37.171778+0000","last_unstale":"2024-04-26T21:08:39.217273+0000","last_undegraded":"2024-04-26T21:08:39.217273+0000","last_fullsized":"2024-04-26T21:08:39.217273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:00.488332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194798+0000","last_change":"2024-04-26T21:08:38.202960+0000","last_active":"2024-04-26T21:08:39.194798+0000","last_peered":"2024-04-26T21:08:39.194798+0000","last_clean":"2024-04-26T21:08:39.194798+0000","last_became_active":"2024-04-26T21:08:38.202642+0000","last_became_peered":"2024-04-26T21:08:38.202642+0000","last_unstale":"2024-04-26T21:08:39.194798+0000","last_undegraded":"2024-04-26T21:08:39.194798+0000","last_fullsized":"2024-04-26T21:08:39.194798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:55.797399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194905+0000","last_change":"2024-04-26T21:08:37.174576+0000","last_active":"2024-04-26T21:08:39.194905+0000","last_peered":"2024-04-26T21:08:39.194905+0000","last_clean":"2024-04-26T21:08:39.194905+0000","last_became_active":"2024-04-26T21:08:37.174223+0000","last_became_peered":"2024-04-26T21:08:37.174223+0000","last_unstale":"2024-04-26T21:08:39.194905+0000","last_undegraded":"2024-04-26T21:08:39.194905+0000","last_fullsized":"2024-04-26T21:08:39.194905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:00.488332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712374+0000","last_change":"2024-04-26T21:08:38.210232+0000","last_active":"2024-04-26T21:08:39.712374+0000","last_peered":"2024-04-26T21:08:39.712374+0000","last_clean":"2024-04-26T21:08:39.712374+0000","last_became_active":"2024-04-26T21:08:38.207957+0000","last_became_peered":"2024-04-26T21:08:38.207957+0000","last_unstale":"2024-04-26T21:08:39.712374+0000","last_undegraded":"2024-04-26T21:08:39.712374+0000","last_fullsized":"2024-04-26T21:08:39.712374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:25:29.659217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211019+0000","last_change":"2024-04-26T21:08:37.166766+0000","last_active":"2024-04-26T21:08:39.211019+0000","last_peered":"2024-04-26T21:08:39.211019+0000","last_clean":"2024-04-26T21:08:39.211019+0000","last_became_active":"2024-04-26T21:08:37.166441+0000","last_became_peered":"2024-04-26T21:08:37.166441+0000","last_unstale":"2024-04-26T21:08:39.211019+0000","last_undegraded":"2024-04-26T21:08:39.211019+0000","last_fullsized":"2024-04-26T21:08:39.211019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:58.240486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195784+0000","last_change":"2024-04-26T21:08:38.209109+0000","last_active":"2024-04-26T21:08:39.195784+0000","last_peered":"2024-04-26T21:08:39.195784+0000","last_clean":"2024-04-26T21:08:39.195784+0000","last_became_active":"2024-04-26T21:08:38.208779+0000","last_became_peered":"2024-04-26T21:08:38.208779+0000","last_unstale":"2024-04-26T21:08:39.195784+0000","last_undegraded":"2024-04-26T21:08:39.195784+0000","last_fullsized":"2024-04-26T21:08:39.195784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:59.255528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217053+0000","last_change":"2024-04-26T21:08:37.176062+0000","last_active":"2024-04-26T21:08:39.217053+0000","last_peered":"2024-04-26T21:08:39.217053+0000","last_clean":"2024-04-26T21:08:39.217053+0000","last_became_active":"2024-04-26T21:08:37.175757+0000","last_became_peered":"2024-04-26T21:08:37.175757+0000","last_unstale":"2024-04-26T21:08:39.217053+0000","last_undegraded":"2024-04-26T21:08:39.217053+0000","last_fullsized":"2024-04-26T21:08:39.217053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:58.240486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711341+0000","last_change":"2024-04-26T21:08:38.199662+0000","last_active":"2024-04-26T21:08:39.711341+0000","last_peered":"2024-04-26T21:08:39.711341+0000","last_clean":"2024-04-26T21:08:39.711341+0000","last_became_active":"2024-04-26T21:08:38.198909+0000","last_became_peered":"2024-04-26T21:08:38.198909+0000","last_unstale":"2024-04-26T21:08:39.711341+0000","last_undegraded":"2024-04-26T21:08:39.711341+0000","last_fullsized":"2024-04-26T21:08:39.711341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:46.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.713546+0000","last_change":"2024-04-26T21:08:37.149949+0000","last_active":"2024-04-26T21:08:39.713546+0000","last_peered":"2024-04-26T21:08:39.713546+0000","last_clean":"2024-04-26T21:08:39.713546+0000","last_became_active":"2024-04-26T21:08:37.149579+0000","last_became_peered":"2024-04-26T21:08:37.149579+0000","last_unstale":"2024-04-26T21:08:39.713546+0000","last_undegraded":"2024-04-26T21:08:39.713546+0000","last_fullsized":"2024-04-26T21:08:39.713546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:29.045350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712396+0000","last_change":"2024-04-26T21:08:38.196010+0000","last_active":"2024-04-26T21:08:39.712396+0000","last_peered":"2024-04-26T21:08:39.712396+0000","last_clean":"2024-04-26T21:08:39.712396+0000","last_became_active":"2024-04-26T21:08:38.195650+0000","last_became_peered":"2024-04-26T21:08:38.195650+0000","last_unstale":"2024-04-26T21:08:39.712396+0000","last_undegraded":"2024-04-26T21:08:39.712396+0000","last_fullsized":"2024-04-26T21:08:39.712396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:59.807950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194744+0000","last_change":"2024-04-26T21:08:37.170877+0000","last_active":"2024-04-26T21:08:39.194744+0000","last_peered":"2024-04-26T21:08:39.194744+0000","last_clean":"2024-04-26T21:08:39.194744+0000","last_became_active":"2024-04-26T21:08:37.170548+0000","last_became_peered":"2024-04-26T21:08:37.170548+0000","last_unstale":"2024-04-26T21:08:39.194744+0000","last_undegraded":"2024-04-26T21:08:39.194744+0000","last_fullsized":"2024-04-26T21:08:39.194744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:58.240486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711189+0000","last_change":"2024-04-26T21:08:38.217551+0000","last_active":"2024-04-26T21:08:39.711189+0000","last_peered":"2024-04-26T21:08:39.711189+0000","last_clean":"2024-04-26T21:08:39.711189+0000","last_became_active":"2024-04-26T21:08:38.217228+0000","last_became_peered":"2024-04-26T21:08:38.217228+0000","last_unstale":"2024-04-26T21:08:39.711189+0000","last_undegraded":"2024-04-26T21:08:39.711189+0000","last_fullsized":"2024-04-26T21:08:39.711189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:00.305226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.216714+0000","last_change":"2024-04-26T21:08:37.170243+0000","last_active":"2024-04-26T21:08:39.216714+0000","last_peered":"2024-04-26T21:08:39.216714+0000","last_clean":"2024-04-26T21:08:39.216714+0000","last_became_active":"2024-04-26T21:08:37.169995+0000","last_became_peered":"2024-04-26T21:08:37.169995+0000","last_unstale":"2024-04-26T21:08:39.216714+0000","last_undegraded":"2024-04-26T21:08:39.216714+0000","last_fullsized":"2024-04-26T21:08:39.216714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:18.839848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.216725+0000","last_change":"2024-04-26T21:08:38.200818+0000","last_active":"2024-04-26T21:08:39.216725+0000","last_peered":"2024-04-26T21:08:39.216725+0000","last_clean":"2024-04-26T21:08:39.216725+0000","last_became_active":"2024-04-26T21:08:38.198256+0000","last_became_peered":"2024-04-26T21:08:38.198256+0000","last_unstale":"2024-04-26T21:08:39.216725+0000","last_undegraded":"2024-04-26T21:08:39.216725+0000","last_fullsized":"2024-04-26T21:08:39.216725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:01.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.191066+0000","last_change":"2024-04-26T21:08:37.170807+0000","last_active":"2024-04-26T21:08:39.191066+0000","last_peered":"2024-04-26T21:08:39.191066+0000","last_clean":"2024-04-26T21:08:39.191066+0000","last_became_active":"2024-04-26T21:08:37.170428+0000","last_became_peered":"2024-04-26T21:08:37.170428+0000","last_unstale":"2024-04-26T21:08:39.191066+0000","last_undegraded":"2024-04-26T21:08:39.191066+0000","last_fullsized":"2024-04-26T21:08:39.191066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:34.683220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711663+0000","last_change":"2024-04-26T21:08:38.209564+0000","last_active":"2024-04-26T21:08:39.711663+0000","last_peered":"2024-04-26T21:08:39.711663+0000","last_clean":"2024-04-26T21:08:39.711663+0000","last_became_active":"2024-04-26T21:08:38.209194+0000","last_became_peered":"2024-04-26T21:08:38.209194+0000","last_unstale":"2024-04-26T21:08:39.711663+0000","last_undegraded":"2024-04-26T21:08:39.711663+0000","last_fullsized":"2024-04-26T21:08:39.711663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:35.132498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.196061+0000","last_change":"2024-04-26T21:08:37.156599+0000","last_active":"2024-04-26T21:08:39.196061+0000","last_peered":"2024-04-26T21:08:39.196061+0000","last_clean":"2024-04-26T21:08:39.196061+0000","last_became_active":"2024-04-26T21:08:37.154195+0000","last_became_peered":"2024-04-26T21:08:37.154195+0000","last_unstale":"2024-04-26T21:08:39.196061+0000","last_undegraded":"2024-04-26T21:08:39.196061+0000","last_fullsized":"2024-04-26T21:08:39.196061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:00.067702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711404+0000","last_change":"2024-04-26T21:08:38.209445+0000","last_active":"2024-04-26T21:08:39.711404+0000","last_peered":"2024-04-26T21:08:39.711404+0000","last_clean":"2024-04-26T21:08:39.711404+0000","last_became_active":"2024-04-26T21:08:38.209134+0000","last_became_peered":"2024-04-26T21:08:38.209134+0000","last_unstale":"2024-04-26T21:08:39.711404+0000","last_undegraded":"2024-04-26T21:08:39.711404+0000","last_fullsized":"2024-04-26T21:08:39.711404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:44.324217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.191281+0000","last_change":"2024-04-26T21:08:37.176762+0000","last_active":"2024-04-26T21:08:39.191281+0000","last_peered":"2024-04-26T21:08:39.191281+0000","last_clean":"2024-04-26T21:08:39.191281+0000","last_became_active":"2024-04-26T21:08:37.176369+0000","last_became_peered":"2024-04-26T21:08:37.176369+0000","last_unstale":"2024-04-26T21:08:39.191281+0000","last_undegraded":"2024-04-26T21:08:39.191281+0000","last_fullsized":"2024-04-26T21:08:39.191281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:00.488332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712217+0000","last_change":"2024-04-26T21:08:38.211784+0000","last_active":"2024-04-26T21:08:39.712217+0000","last_peered":"2024-04-26T21:08:39.712217+0000","last_clean":"2024-04-26T21:08:39.712217+0000","last_became_active":"2024-04-26T21:08:38.210121+0000","last_became_peered":"2024-04-26T21:08:38.210121+0000","last_unstale":"2024-04-26T21:08:39.712217+0000","last_undegraded":"2024-04-26T21:08:39.712217+0000","last_fullsized":"2024-04-26T21:08:39.712217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:46.194523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211130+0000","last_change":"2024-04-26T21:08:37.167350+0000","last_active":"2024-04-26T21:08:39.211130+0000","last_peered":"2024-04-26T21:08:39.211130+0000","last_clean":"2024-04-26T21:08:39.211130+0000","last_became_active":"2024-04-26T21:08:37.166952+0000","last_became_peered":"2024-04-26T21:08:37.166952+0000","last_unstale":"2024-04-26T21:08:39.211130+0000","last_undegraded":"2024-04-26T21:08:39.211130+0000","last_fullsized":"2024-04-26T21:08:39.211130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:30.628461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195797+0000","last_change":"2024-04-26T21:08:38.202682+0000","last_active":"2024-04-26T21:08:39.195797+0000","last_peered":"2024-04-26T21:08:39.195797+0000","last_clean":"2024-04-26T21:08:39.195797+0000","last_became_active":"2024-04-26T21:08:38.202299+0000","last_became_peered":"2024-04-26T21:08:38.202299+0000","last_unstale":"2024-04-26T21:08:39.195797+0000","last_undegraded":"2024-04-26T21:08:39.195797+0000","last_fullsized":"2024-04-26T21:08:39.195797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:19.756896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.191430+0000","last_change":"2024-04-26T21:08:37.168687+0000","last_active":"2024-04-26T21:08:39.191430+0000","last_peered":"2024-04-26T21:08:39.191430+0000","last_clean":"2024-04-26T21:08:39.191430+0000","last_became_active":"2024-04-26T21:08:37.168302+0000","last_became_peered":"2024-04-26T21:08:37.168302+0000","last_unstale":"2024-04-26T21:08:39.191430+0000","last_undegraded":"2024-04-26T21:08:39.191430+0000","last_fullsized":"2024-04-26T21:08:39.191430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:48:02.545851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711826+0000","last_change":"2024-04-26T21:08:38.208610+0000","last_active":"2024-04-26T21:08:39.711826+0000","last_peered":"2024-04-26T21:08:39.711826+0000","last_clean":"2024-04-26T21:08:39.711826+0000","last_became_active":"2024-04-26T21:08:38.206735+0000","last_became_peered":"2024-04-26T21:08:38.206735+0000","last_unstale":"2024-04-26T21:08:39.711826+0000","last_undegraded":"2024-04-26T21:08:39.711826+0000","last_fullsized":"2024-04-26T21:08:39.711826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:54.952928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.710746+0000","last_change":"2024-04-26T21:08:33.504911+0000","last_active":"2024-04-26T21:08:39.710746+0000","last_peered":"2024-04-26T21:08:39.710746+0000","last_clean":"2024-04-26T21:08:39.710746+0000","last_became_active":"2024-04-26T21:08:33.503667+0000","last_became_peered":"2024-04-26T21:08:33.503667+0000","last_unstale":"2024-04-26T21:08:39.710746+0000","last_undegraded":"2024-04-26T21:08:39.710746+0000","last_fullsized":"2024-04-26T21:08:39.710746+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:32.013907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:32.013907+0000","last_clean_scrub_stamp":"2024-04-26T21:08:32.013907+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:09.596153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194166+0000","last_change":"2024-04-26T21:08:37.161497+0000","last_active":"2024-04-26T21:08:39.194166+0000","last_peered":"2024-04-26T21:08:39.194166+0000","last_clean":"2024-04-26T21:08:39.194166+0000","last_became_active":"2024-04-26T21:08:37.161144+0000","last_became_peered":"2024-04-26T21:08:37.161144+0000","last_unstale":"2024-04-26T21:08:39.194166+0000","last_undegraded":"2024-04-26T21:08:39.194166+0000","last_fullsized":"2024-04-26T21:08:39.194166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:18.839848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712097+0000","last_change":"2024-04-26T21:08:38.211227+0000","last_active":"2024-04-26T21:08:39.712097+0000","last_peered":"2024-04-26T21:08:39.712097+0000","last_clean":"2024-04-26T21:08:39.712097+0000","last_became_active":"2024-04-26T21:08:38.210882+0000","last_became_peered":"2024-04-26T21:08:38.210882+0000","last_unstale":"2024-04-26T21:08:39.712097+0000","last_undegraded":"2024-04-26T21:08:39.712097+0000","last_fullsized":"2024-04-26T21:08:39.712097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:42.506444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.710561+0000","last_change":"2024-04-26T21:08:37.177216+0000","last_active":"2024-04-26T21:08:39.710561+0000","last_peered":"2024-04-26T21:08:39.710561+0000","last_clean":"2024-04-26T21:08:39.710561+0000","last_became_active":"2024-04-26T21:08:37.176919+0000","last_became_peered":"2024-04-26T21:08:37.176919+0000","last_unstale":"2024-04-26T21:08:39.710561+0000","last_undegraded":"2024-04-26T21:08:39.710561+0000","last_fullsized":"2024-04-26T21:08:39.710561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:58.240486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712236+0000","last_change":"2024-04-26T21:08:38.191994+0000","last_active":"2024-04-26T21:08:39.712236+0000","last_peered":"2024-04-26T21:08:39.712236+0000","last_clean":"2024-04-26T21:08:39.712236+0000","last_became_active":"2024-04-26T21:08:38.191567+0000","last_became_peered":"2024-04-26T21:08:38.191567+0000","last_unstale":"2024-04-26T21:08:39.712236+0000","last_undegraded":"2024-04-26T21:08:39.712236+0000","last_fullsized":"2024-04-26T21:08:39.712236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:01.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.191022+0000","last_change":"2024-04-26T21:08:37.172777+0000","last_active":"2024-04-26T21:08:39.191022+0000","last_peered":"2024-04-26T21:08:39.191022+0000","last_clean":"2024-04-26T21:08:39.191022+0000","last_became_active":"2024-04-26T21:08:37.171865+0000","last_became_peered":"2024-04-26T21:08:37.171865+0000","last_unstale":"2024-04-26T21:08:39.191022+0000","last_undegraded":"2024-04-26T21:08:39.191022+0000","last_fullsized":"2024-04-26T21:08:39.191022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:18.839848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.210976+0000","last_change":"2024-04-26T21:08:38.200547+0000","last_active":"2024-04-26T21:08:39.210976+0000","last_peered":"2024-04-26T21:08:39.210976+0000","last_clean":"2024-04-26T21:08:39.210976+0000","last_became_active":"2024-04-26T21:08:38.200025+0000","last_became_peered":"2024-04-26T21:08:38.200025+0000","last_unstale":"2024-04-26T21:08:39.210976+0000","last_undegraded":"2024-04-26T21:08:39.210976+0000","last_fullsized":"2024-04-26T21:08:39.210976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:59.807950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194881+0000","last_change":"2024-04-26T21:08:37.170761+0000","last_active":"2024-04-26T21:08:39.194881+0000","last_peered":"2024-04-26T21:08:39.194881+0000","last_clean":"2024-04-26T21:08:39.194881+0000","last_became_active":"2024-04-26T21:08:37.170431+0000","last_became_peered":"2024-04-26T21:08:37.170431+0000","last_unstale":"2024-04-26T21:08:39.194881+0000","last_undegraded":"2024-04-26T21:08:39.194881+0000","last_fullsized":"2024-04-26T21:08:39.194881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:59.977930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195387+0000","last_change":"2024-04-26T21:08:38.213352+0000","last_active":"2024-04-26T21:08:39.195387+0000","last_peered":"2024-04-26T21:08:39.195387+0000","last_clean":"2024-04-26T21:08:39.195387+0000","last_became_active":"2024-04-26T21:08:38.201057+0000","last_became_peered":"2024-04-26T21:08:38.201057+0000","last_unstale":"2024-04-26T21:08:39.195387+0000","last_undegraded":"2024-04-26T21:08:39.195387+0000","last_fullsized":"2024-04-26T21:08:39.195387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:35.698263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.210868+0000","last_change":"2024-04-26T21:08:37.169249+0000","last_active":"2024-04-26T21:08:39.210868+0000","last_peered":"2024-04-26T21:08:39.210868+0000","last_clean":"2024-04-26T21:08:39.210868+0000","last_became_active":"2024-04-26T21:08:37.168910+0000","last_became_peered":"2024-04-26T21:08:37.168910+0000","last_unstale":"2024-04-26T21:08:39.210868+0000","last_undegraded":"2024-04-26T21:08:39.210868+0000","last_fullsized":"2024-04-26T21:08:39.210868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:34.683220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194515+0000","last_change":"2024-04-26T21:08:38.206722+0000","last_active":"2024-04-26T21:08:39.194515+0000","last_peered":"2024-04-26T21:08:39.194515+0000","last_clean":"2024-04-26T21:08:39.194515+0000","last_became_active":"2024-04-26T21:08:38.205394+0000","last_became_peered":"2024-04-26T21:08:38.205394+0000","last_unstale":"2024-04-26T21:08:39.194515+0000","last_undegraded":"2024-04-26T21:08:39.194515+0000","last_fullsized":"2024-04-26T21:08:39.194515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:44.324217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712111+0000","last_change":"2024-04-26T21:08:37.178417+0000","last_active":"2024-04-26T21:08:39.712111+0000","last_peered":"2024-04-26T21:08:39.712111+0000","last_clean":"2024-04-26T21:08:39.712111+0000","last_became_active":"2024-04-26T21:08:37.178031+0000","last_became_peered":"2024-04-26T21:08:37.178031+0000","last_unstale":"2024-04-26T21:08:39.712111+0000","last_undegraded":"2024-04-26T21:08:39.712111+0000","last_fullsized":"2024-04-26T21:08:39.712111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:54.872853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.210824+0000","last_change":"2024-04-26T21:08:38.200358+0000","last_active":"2024-04-26T21:08:39.210824+0000","last_peered":"2024-04-26T21:08:39.210824+0000","last_clean":"2024-04-26T21:08:39.210824+0000","last_became_active":"2024-04-26T21:08:38.200001+0000","last_became_peered":"2024-04-26T21:08:38.200001+0000","last_unstale":"2024-04-26T21:08:39.210824+0000","last_undegraded":"2024-04-26T21:08:39.210824+0000","last_fullsized":"2024-04-26T21:08:39.210824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:01.330449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.215005+0000","last_change":"2024-04-26T21:08:37.178473+0000","last_active":"2024-04-26T21:08:39.215005+0000","last_peered":"2024-04-26T21:08:39.215005+0000","last_clean":"2024-04-26T21:08:39.215005+0000","last_became_active":"2024-04-26T21:08:37.178079+0000","last_became_peered":"2024-04-26T21:08:37.178079+0000","last_unstale":"2024-04-26T21:08:39.215005+0000","last_undegraded":"2024-04-26T21:08:39.215005+0000","last_fullsized":"2024-04-26T21:08:39.215005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:31.424853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712508+0000","last_change":"2024-04-26T21:08:38.212830+0000","last_active":"2024-04-26T21:08:39.712508+0000","last_peered":"2024-04-26T21:08:39.712508+0000","last_clean":"2024-04-26T21:08:39.712508+0000","last_became_active":"2024-04-26T21:08:38.212327+0000","last_became_peered":"2024-04-26T21:08:38.212327+0000","last_unstale":"2024-04-26T21:08:39.712508+0000","last_undegraded":"2024-04-26T21:08:39.712508+0000","last_fullsized":"2024-04-26T21:08:39.712508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:26.983477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217434+0000","last_change":"2024-04-26T21:08:37.172503+0000","last_active":"2024-04-26T21:08:39.217434+0000","last_peered":"2024-04-26T21:08:39.217434+0000","last_clean":"2024-04-26T21:08:39.217434+0000","last_became_active":"2024-04-26T21:08:37.172129+0000","last_became_peered":"2024-04-26T21:08:37.172129+0000","last_unstale":"2024-04-26T21:08:39.217434+0000","last_undegraded":"2024-04-26T21:08:39.217434+0000","last_fullsized":"2024-04-26T21:08:39.217434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:52:34.683220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712025+0000","last_change":"2024-04-26T21:08:38.192302+0000","last_active":"2024-04-26T21:08:39.712025+0000","last_peered":"2024-04-26T21:08:39.712025+0000","last_clean":"2024-04-26T21:08:39.712025+0000","last_became_active":"2024-04-26T21:08:38.191741+0000","last_became_peered":"2024-04-26T21:08:38.191741+0000","last_unstale":"2024-04-26T21:08:39.712025+0000","last_undegraded":"2024-04-26T21:08:39.712025+0000","last_fullsized":"2024-04-26T21:08:39.712025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:48.443740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.710597+0000","last_change":"2024-04-26T21:08:37.178123+0000","last_active":"2024-04-26T21:08:39.710597+0000","last_peered":"2024-04-26T21:08:39.710597+0000","last_clean":"2024-04-26T21:08:39.710597+0000","last_became_active":"2024-04-26T21:08:37.177831+0000","last_became_peered":"2024-04-26T21:08:37.177831+0000","last_unstale":"2024-04-26T21:08:39.710597+0000","last_undegraded":"2024-04-26T21:08:39.710597+0000","last_fullsized":"2024-04-26T21:08:39.710597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:59.977930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.196679+0000","last_change":"2024-04-26T21:08:38.212239+0000","last_active":"2024-04-26T21:08:39.196679+0000","last_peered":"2024-04-26T21:08:39.196679+0000","last_clean":"2024-04-26T21:08:39.196679+0000","last_became_active":"2024-04-26T21:08:38.209717+0000","last_became_peered":"2024-04-26T21:08:38.209717+0000","last_unstale":"2024-04-26T21:08:39.196679+0000","last_undegraded":"2024-04-26T21:08:39.196679+0000","last_fullsized":"2024-04-26T21:08:39.196679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:31.643503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211322+0000","last_change":"2024-04-26T21:08:37.173887+0000","last_active":"2024-04-26T21:08:39.211322+0000","last_peered":"2024-04-26T21:08:39.211322+0000","last_clean":"2024-04-26T21:08:39.211322+0000","last_became_active":"2024-04-26T21:08:37.173034+0000","last_became_peered":"2024-04-26T21:08:37.173034+0000","last_unstale":"2024-04-26T21:08:39.211322+0000","last_undegraded":"2024-04-26T21:08:39.211322+0000","last_fullsized":"2024-04-26T21:08:39.211322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:00.488332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712910+0000","last_change":"2024-04-26T21:08:38.197901+0000","last_active":"2024-04-26T21:08:39.712910+0000","last_peered":"2024-04-26T21:08:39.712910+0000","last_clean":"2024-04-26T21:08:39.712910+0000","last_became_active":"2024-04-26T21:08:38.197592+0000","last_became_peered":"2024-04-26T21:08:38.197592+0000","last_unstale":"2024-04-26T21:08:39.712910+0000","last_undegraded":"2024-04-26T21:08:39.712910+0000","last_fullsized":"2024-04-26T21:08:39.712910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:46.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.211038+0000","last_change":"2024-04-26T21:08:37.175903+0000","last_active":"2024-04-26T21:08:39.211038+0000","last_peered":"2024-04-26T21:08:39.211038+0000","last_clean":"2024-04-26T21:08:39.211038+0000","last_became_active":"2024-04-26T21:08:37.175605+0000","last_became_peered":"2024-04-26T21:08:37.175605+0000","last_unstale":"2024-04-26T21:08:39.211038+0000","last_undegraded":"2024-04-26T21:08:39.211038+0000","last_fullsized":"2024-04-26T21:08:39.211038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:59.977930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711599+0000","last_change":"2024-04-26T21:08:38.208922+0000","last_active":"2024-04-26T21:08:39.711599+0000","last_peered":"2024-04-26T21:08:39.711599+0000","last_clean":"2024-04-26T21:08:39.711599+0000","last_became_active":"2024-04-26T21:08:38.208597+0000","last_became_peered":"2024-04-26T21:08:38.208597+0000","last_unstale":"2024-04-26T21:08:39.711599+0000","last_undegraded":"2024-04-26T21:08:39.711599+0000","last_fullsized":"2024-04-26T21:08:39.711599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:59.807950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711492+0000","last_change":"2024-04-26T21:08:37.173916+0000","last_active":"2024-04-26T21:08:39.711492+0000","last_peered":"2024-04-26T21:08:39.711492+0000","last_clean":"2024-04-26T21:08:39.711492+0000","last_became_active":"2024-04-26T21:08:37.173553+0000","last_became_peered":"2024-04-26T21:08:37.173553+0000","last_unstale":"2024-04-26T21:08:39.711492+0000","last_undegraded":"2024-04-26T21:08:39.711492+0000","last_fullsized":"2024-04-26T21:08:39.711492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:28:30.628461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.217095+0000","last_change":"2024-04-26T21:08:38.212699+0000","last_active":"2024-04-26T21:08:39.217095+0000","last_peered":"2024-04-26T21:08:39.217095+0000","last_clean":"2024-04-26T21:08:39.217095+0000","last_became_active":"2024-04-26T21:08:38.212418+0000","last_became_peered":"2024-04-26T21:08:38.212418+0000","last_unstale":"2024-04-26T21:08:39.217095+0000","last_undegraded":"2024-04-26T21:08:39.217095+0000","last_fullsized":"2024-04-26T21:08:39.217095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:11:59.807950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.194366+0000","last_change":"2024-04-26T21:08:38.211036+0000","last_active":"2024-04-26T21:08:39.194366+0000","last_peered":"2024-04-26T21:08:39.194366+0000","last_clean":"2024-04-26T21:08:39.194366+0000","last_became_active":"2024-04-26T21:08:38.207129+0000","last_became_peered":"2024-04-26T21:08:38.207129+0000","last_unstale":"2024-04-26T21:08:39.194366+0000","last_undegraded":"2024-04-26T21:08:39.194366+0000","last_fullsized":"2024-04-26T21:08:39.194366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:38:46.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711252+0000","last_change":"2024-04-26T21:08:37.179811+0000","last_active":"2024-04-26T21:08:39.711252+0000","last_peered":"2024-04-26T21:08:39.711252+0000","last_clean":"2024-04-26T21:08:39.711252+0000","last_became_active":"2024-04-26T21:08:37.179425+0000","last_became_peered":"2024-04-26T21:08:37.179425+0000","last_unstale":"2024-04-26T21:08:39.711252+0000","last_undegraded":"2024-04-26T21:08:39.711252+0000","last_fullsized":"2024-04-26T21:08:39.711252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:31.424853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.195669+0000","last_change":"2024-04-26T21:08:38.207278+0000","last_active":"2024-04-26T21:08:39.195669+0000","last_peered":"2024-04-26T21:08:39.195669+0000","last_clean":"2024-04-26T21:08:39.195669+0000","last_became_active":"2024-04-26T21:08:38.204849+0000","last_became_peered":"2024-04-26T21:08:38.204849+0000","last_unstale":"2024-04-26T21:08:39.195669+0000","last_undegraded":"2024-04-26T21:08:39.195669+0000","last_fullsized":"2024-04-26T21:08:39.195669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:54.952928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711653+0000","last_change":"2024-04-26T21:08:37.175992+0000","last_active":"2024-04-26T21:08:39.711653+0000","last_peered":"2024-04-26T21:08:39.711653+0000","last_clean":"2024-04-26T21:08:39.711653+0000","last_became_active":"2024-04-26T21:08:37.175682+0000","last_became_peered":"2024-04-26T21:08:37.175682+0000","last_unstale":"2024-04-26T21:08:39.711653+0000","last_undegraded":"2024-04-26T21:08:39.711653+0000","last_fullsized":"2024-04-26T21:08:39.711653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:33.374984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.710839+0000","last_change":"2024-04-26T21:08:38.207957+0000","last_active":"2024-04-26T21:08:39.710839+0000","last_peered":"2024-04-26T21:08:39.710839+0000","last_clean":"2024-04-26T21:08:39.710839+0000","last_became_active":"2024-04-26T21:08:38.207573+0000","last_became_peered":"2024-04-26T21:08:38.207573+0000","last_unstale":"2024-04-26T21:08:39.710839+0000","last_undegraded":"2024-04-26T21:08:39.710839+0000","last_fullsized":"2024-04-26T21:08:39.710839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:02:14.307645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712093+0000","last_change":"2024-04-26T21:08:37.173718+0000","last_active":"2024-04-26T21:08:39.712093+0000","last_peered":"2024-04-26T21:08:39.712093+0000","last_clean":"2024-04-26T21:08:39.712093+0000","last_became_active":"2024-04-26T21:08:37.170846+0000","last_became_peered":"2024-04-26T21:08:37.170846+0000","last_unstale":"2024-04-26T21:08:39.712093+0000","last_undegraded":"2024-04-26T21:08:39.712093+0000","last_fullsized":"2024-04-26T21:08:39.712093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:48:02.545851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.711803+0000","last_change":"2024-04-26T21:08:38.199029+0000","last_active":"2024-04-26T21:08:39.711803+0000","last_peered":"2024-04-26T21:08:39.711803+0000","last_clean":"2024-04-26T21:08:39.711803+0000","last_became_active":"2024-04-26T21:08:38.198720+0000","last_became_peered":"2024-04-26T21:08:38.198720+0000","last_unstale":"2024-04-26T21:08:39.711803+0000","last_undegraded":"2024-04-26T21:08:39.711803+0000","last_fullsized":"2024-04-26T21:08:39.711803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:37.123267+0000","last_clean_scrub_stamp":"2024-04-26T21:08:37.123267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:36:40.618646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T21:08:39.712817+0000","last_change":"2024-04-26T21:08:37.165057+0000","last_active":"2024-04-26T21:08:39.712817+0000","last_peered":"2024-04-26T21:08:39.712817+0000","last_clean":"2024-04-26T21:08:39.712817+0000","last_became_active":"2024-04-26T21:08:37.164781+0000","last_became_peered":"2024-04-26T21:08:37.164781+0000","last_unstale":"2024-04-26T21:08:39.712817+0000","last_undegraded":"2024-04-26T21:08:39.712817+0000","last_fullsized":"2024-04-26T21:08:39.712817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:36.108225+0000","last_clean_scrub_stamp":"2024-04-26T21:08:36.108225+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:17.499100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:08:48.745 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T21:08:48.745 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T21:08:48.745 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-26T21:08:48.745 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T21:08:48.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.991+0000 7fb103e00640 1 Processor -- start 2024-04-26T21:08:48.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.991+0000 7fb103e00640 1 -- start start 2024-04-26T21:08:48.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.992+0000 7fb103e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb1040b4ac0 0x7fb10415d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:48.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.993+0000 7fb103e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb10406e730 con 0x7fb10415fac0 2024-04-26T21:08:48.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.993+0000 7fb103e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb104160310 con 0x7fb1040b3670 2024-04-26T21:08:48.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.993+0000 7fb103e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fb1041605f0 con 0x7fb1040b4ac0 2024-04-26T21:08:48.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.993+0000 7fb102a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb1040b4ac0 0x7fb10415d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:48.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.993+0000 7fb102a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb1040b4ac0 0x7fb10415d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34754/0 (socket says 172.21.15.26:34754) 2024-04-26T21:08:48.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.993+0000 7fb102a00640 1 -- 172.21.15.26:0/77833169 learned_addr learned my addr 172.21.15.26:0/77833169 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:48.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.993+0000 7fb102a00640 1 -- 172.21.15.26:0/77833169 >> v1:172.21.15.86:6789/0 conn(0x7fb1040b3670 legacy=0x7fb10406eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:48.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.993+0000 7fb102a00640 1 -- 172.21.15.26:0/77833169 >> v1:172.21.15.26:6789/0 conn(0x7fb10415fac0 legacy=0x7fb1040b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:48.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.993+0000 7fb102a00640 1 -- 172.21.15.26:0/77833169 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10406e9a0 con 0x7fb1040b4ac0 2024-04-26T21:08:48.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.994+0000 7fb102a00640 1 --2- 172.21.15.26:0/77833169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1040b4ac0 0x7fb10415d070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009870 tx=0x7fb0f402a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4411a68cb57d161 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:48.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.995+0000 7fb101600640 1 -- 172.21.15.26:0/77833169 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4037070 con 0x7fb1040b4ac0 2024-04-26T21:08:48.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.995+0000 7fb101600640 1 -- 172.21.15.26:0/77833169 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f4032640 con 0x7fb1040b4ac0 2024-04-26T21:08:48.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.995+0000 7fb101600640 1 -- 172.21.15.26:0/77833169 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4032a10 con 0x7fb1040b4ac0 2024-04-26T21:08:48.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.995+0000 7fb103e00640 1 -- 172.21.15.26:0/77833169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1040b4ac0 msgr2=0x7fb10415d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:48.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.995+0000 7fb103e00640 1 --2- 172.21.15.26:0/77833169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1040b4ac0 0x7fb10415d070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009870 tx=0x7fb0f402a330 comp rx=0 tx=0).stop 2024-04-26T21:08:48.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.995+0000 7fb103e00640 1 -- 172.21.15.26:0/77833169 shutdown_connections 2024-04-26T21:08:48.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.995+0000 7fb103e00640 1 --2- 172.21.15.26:0/77833169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1040b4ac0 0x7fb10415d070 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:48.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.995+0000 7fb103e00640 1 -- 172.21.15.26:0/77833169 >> 172.21.15.26:0/77833169 conn(0x7fb10406c660 msgr2=0x7fb10406ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:48.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.996+0000 7fb103e00640 1 -- 172.21.15.26:0/77833169 shutdown_connections 2024-04-26T21:08:48.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.996+0000 7fb103e00640 1 -- 172.21.15.26:0/77833169 wait complete. 2024-04-26T21:08:48.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.997+0000 7fb103e00640 1 Processor -- start 2024-04-26T21:08:48.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.997+0000 7fb103e00640 1 -- start start 2024-04-26T21:08:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.998+0000 7fb103e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1040b3670 0x7fb104073fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.998+0000 7fb103e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1040b4ac0 0x7fb1041e58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.998+0000 7fb103e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10415fac0 0x7fb10415b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.998+0000 7fb103e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb1040b5020 con 0x7fb10415fac0 2024-04-26T21:08:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.998+0000 7fb103e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1040b4ea0 con 0x7fb1040b3670 2024-04-26T21:08:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.998+0000 7fb103e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb10415ff50 con 0x7fb1040b4ac0 2024-04-26T21:08:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.999+0000 7fb103400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10415fac0 0x7fb10415b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.999+0000 7fb103400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10415fac0 0x7fb10415b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34762/0 (socket says 172.21.15.26:34762) 2024-04-26T21:08:49.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.999+0000 7fb103400640 1 -- 172.21.15.26:0/1110931142 learned_addr learned my addr 172.21.15.26:0/1110931142 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:49.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.999+0000 7fb103400640 1 -- 172.21.15.26:0/1110931142 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1040b4ac0 msgr2=0x7fb1041e58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:49.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.999+0000 7fb103400640 1 --2- 172.21.15.26:0/1110931142 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1040b4ac0 0x7fb1041e58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.999+0000 7fb103400640 1 -- 172.21.15.26:0/1110931142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1040b3670 msgr2=0x7fb104073fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:49.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.999+0000 7fb103400640 1 --2- 172.21.15.26:0/1110931142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1040b3670 0x7fb104073fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.999+0000 7fb103400640 1 -- 172.21.15.26:0/1110931142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1041e63a0 con 0x7fb10415fac0 2024-04-26T21:08:49.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.999+0000 7fb103400640 1 --2- 172.21.15.26:0/1110931142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10415fac0 0x7fb10415b0b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00c700 tx=0x7fb0fc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:49.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.999+0000 7fb0fb400640 1 -- 172.21.15.26:0/1110931142 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc016020 con 0x7fb10415fac0 2024-04-26T21:08:49.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.999+0000 7fb0fb400640 1 -- 172.21.15.26:0/1110931142 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0fc00aa50 con 0x7fb10415fac0 2024-04-26T21:08:49.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:48.999+0000 7fb0fb400640 1 -- 172.21.15.26:0/1110931142 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc00ce10 con 0x7fb10415fac0 2024-04-26T21:08:49.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.000+0000 7fb103e00640 1 -- 172.21.15.26:0/1110931142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1041e6680 con 0x7fb10415fac0 2024-04-26T21:08:49.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.000+0000 7fb103e00640 1 -- 172.21.15.26:0/1110931142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1041e6c40 con 0x7fb10415fac0 2024-04-26T21:08:49.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.001+0000 7fb103e00640 1 -- 172.21.15.26:0/1110931142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb10406a700 con 0x7fb10415fac0 2024-04-26T21:08:49.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.002+0000 7fb0fb400640 1 -- 172.21.15.26:0/1110931142 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0fc019020 con 0x7fb10415fac0 2024-04-26T21:08:49.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.003+0000 7fb0fb400640 1 --2- 172.21.15.26:0/1110931142 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb0e406c8f0 0x7fb0e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:49.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.003+0000 7fb0fb400640 1 -- 172.21.15.26:0/1110931142 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb0fc08be80 con 0x7fb10415fac0 2024-04-26T21:08:49.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.003+0000 7fb102a00640 1 --2- 172.21.15.26:0/1110931142 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb0e406c8f0 0x7fb0e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:49.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.004+0000 7fb102a00640 1 --2- 172.21.15.26:0/1110931142 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb0e406c8f0 0x7fb0e406edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4034000 tx=0x7fb0f402c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:49.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.007+0000 7fb0fb400640 1 -- 172.21.15.26:0/1110931142 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb0fc020020 con 0x7fb10415fac0 2024-04-26T21:08:49.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.246+0000 7fb103e00640 1 -- 172.21.15.26:0/1110931142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb10406cdc0 con 0x7fb10415fac0 2024-04-26T21:08:49.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.247+0000 7fb0fb400640 1 -- 172.21.15.26:0/1110931142 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb0fc05a2a0 con 0x7fb10415fac0 2024-04-26T21:08:49.247 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:49.247 INFO:teuthology.orchestra.run.smithi026.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T21:08:49.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.249+0000 7fb103e00640 1 -- 172.21.15.26:0/1110931142 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb0e406c8f0 msgr2=0x7fb0e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:49.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.249+0000 7fb103e00640 1 --2- 172.21.15.26:0/1110931142 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb0e406c8f0 0x7fb0e406edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4034000 tx=0x7fb0f402c000 comp rx=0 tx=0).stop 2024-04-26T21:08:49.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.249+0000 7fb103e00640 1 -- 172.21.15.26:0/1110931142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10415fac0 msgr2=0x7fb10415b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:49.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.249+0000 7fb103e00640 1 --2- 172.21.15.26:0/1110931142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10415fac0 0x7fb10415b0b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00c700 tx=0x7fb0fc00cbc0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.250+0000 7fb103e00640 1 -- 172.21.15.26:0/1110931142 shutdown_connections 2024-04-26T21:08:49.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.250+0000 7fb103e00640 1 --2- 172.21.15.26:0/1110931142 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb0e406c8f0 0x7fb0e406edb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.250+0000 7fb103e00640 1 --2- 172.21.15.26:0/1110931142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10415fac0 0x7fb10415b0b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.250+0000 7fb103e00640 1 --2- 172.21.15.26:0/1110931142 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1040b4ac0 0x7fb1041e58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.250+0000 7fb103e00640 1 --2- 172.21.15.26:0/1110931142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1040b3670 0x7fb104073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.250+0000 7fb103e00640 1 -- 172.21.15.26:0/1110931142 >> 172.21.15.26:0/1110931142 conn(0x7fb10406c660 msgr2=0x7fb1040b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:49.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.250+0000 7fb103e00640 1 -- 172.21.15.26:0/1110931142 shutdown_connections 2024-04-26T21:08:49.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.250+0000 7fb103e00640 1 -- 172.21.15.26:0/1110931142 wait complete. 2024-04-26T21:08:49.269 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-26T21:08:49.269 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:08:49.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.476+0000 7faee9e00640 1 Processor -- start 2024-04-26T21:08:49.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.476+0000 7faee9e00640 1 -- start start 2024-04-26T21:08:49.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.477+0000 7faee9e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7faee415f680 0x7faee40d5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:49.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.477+0000 7faee9e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faee4160500 0x7faee40d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:49.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.477+0000 7faee9e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faee40d6110 con 0x7faee40d1b80 2024-04-26T21:08:49.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.477+0000 7faee9e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7faee40d0870 con 0x7faee415f680 2024-04-26T21:08:49.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.477+0000 7faee9e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7faee40d1240 con 0x7faee4160500 2024-04-26T21:08:49.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.478+0000 7faee2000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faee4160500 0x7faee40d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:49.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.478+0000 7faee2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7faee415f680 0x7faee40d5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:49.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.478+0000 7faee3400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7faee40d1b80 0x7faee40d2d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.26:57192/0 (socket says 172.21.15.26:57192) 2024-04-26T21:08:49.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.478+0000 7faee3400640 1 -- 172.21.15.26:0/1985354162 learned_addr learned my addr 172.21.15.26:0/1985354162 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:49.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.478+0000 7faee1600640 1 -- 172.21.15.26:0/1985354162 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 368629383 0 0) 0x7faee40d6110 con 0x7faee40d1b80 2024-04-26T21:08:49.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.478+0000 7faee1600640 1 -- 172.21.15.26:0/1985354162 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faec8003520 con 0x7faee40d1b80 2024-04-26T21:08:49.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.479+0000 7faee2a00640 1 -- 172.21.15.26:0/1985354162 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faee4160500 msgr2=0x7faee40d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:49.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.479+0000 7faee2a00640 1 --2- 172.21.15.26:0/1985354162 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faee4160500 0x7faee40d0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.479+0000 7faee2a00640 1 -- 172.21.15.26:0/1985354162 >> v1:172.21.15.86:6789/0 conn(0x7faee40d1b80 legacy=0x7faee40d2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:49.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.479+0000 7faee2a00640 1 -- 172.21.15.26:0/1985354162 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee40cff40 con 0x7faee415f680 2024-04-26T21:08:49.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.479+0000 7faee2000640 1 --2- 172.21.15.26:0/1985354162 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faee4160500 0x7faee40d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:08:49.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.480+0000 7faee2a00640 1 --2- 172.21.15.26:0/1985354162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faee415f680 0x7faee40d5ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faed40097a0 tx=0x7faed402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=856ec018be5264fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:49.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.480+0000 7faee1600640 1 -- 172.21.15.26:0/1985354162 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4037070 con 0x7faee415f680 2024-04-26T21:08:49.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.480+0000 7faee1600640 1 -- 172.21.15.26:0/1985354162 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed4032440 con 0x7faee415f680 2024-04-26T21:08:49.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.480+0000 7faee1600640 1 -- 172.21.15.26:0/1985354162 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4032810 con 0x7faee415f680 2024-04-26T21:08:49.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.481+0000 7faee9e00640 1 -- 172.21.15.26:0/1985354162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faee415f680 msgr2=0x7faee40d5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:49.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.481+0000 7faee9e00640 1 --2- 172.21.15.26:0/1985354162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faee415f680 0x7faee40d5ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faed40097a0 tx=0x7faed402a180 comp rx=0 tx=0).stop 2024-04-26T21:08:49.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.481+0000 7faee9e00640 1 -- 172.21.15.26:0/1985354162 shutdown_connections 2024-04-26T21:08:49.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.481+0000 7faee9e00640 1 --2- 172.21.15.26:0/1985354162 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faee4160500 0x7faee40d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.481+0000 7faee9e00640 1 --2- 172.21.15.26:0/1985354162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faee415f680 0x7faee40d5ae0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.481+0000 7faee9e00640 1 -- 172.21.15.26:0/1985354162 >> 172.21.15.26:0/1985354162 conn(0x7faee409e490 msgr2=0x7faee40d51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:49.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.481+0000 7faee9e00640 1 -- 172.21.15.26:0/1985354162 shutdown_connections 2024-04-26T21:08:49.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.481+0000 7faee9e00640 1 -- 172.21.15.26:0/1985354162 wait complete. 2024-04-26T21:08:49.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.483+0000 7faee9e00640 1 Processor -- start 2024-04-26T21:08:49.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.483+0000 7faee9e00640 1 -- start start 2024-04-26T21:08:49.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.484+0000 7faee9e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faee40d1b80 0x7faee4159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:49.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.484+0000 7faee9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faee415f680 0x7faee40ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:49.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.484+0000 7faee9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faee4160500 0x7faee41617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:49.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.484+0000 7faee9e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faee4160940 con 0x7faee415f680 2024-04-26T21:08:49.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.484+0000 7faee9e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faee40d0500 con 0x7faee4160500 2024-04-26T21:08:49.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.484+0000 7faee9e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7faee40d2010 con 0x7faee40d1b80 2024-04-26T21:08:49.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.485+0000 7faee2000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faee415f680 0x7faee40ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:49.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.485+0000 7faee2000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faee415f680 0x7faee40ce5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34774/0 (socket says 172.21.15.26:34774) 2024-04-26T21:08:49.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.485+0000 7faee2000640 1 -- 172.21.15.26:0/1488650648 learned_addr learned my addr 172.21.15.26:0/1488650648 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:49.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.485+0000 7faee2a00640 1 --2- 172.21.15.26:0/1488650648 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faee40d1b80 0x7faee4159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:49.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.485+0000 7faee3400640 1 --2- 172.21.15.26:0/1488650648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faee4160500 0x7faee41617c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:49.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.485+0000 7faee2000640 1 -- 172.21.15.26:0/1488650648 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faee40d1b80 msgr2=0x7faee4159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:49.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.485+0000 7faee2000640 1 --2- 172.21.15.26:0/1488650648 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faee40d1b80 0x7faee4159560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.485+0000 7faee2000640 1 -- 172.21.15.26:0/1488650648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faee4160500 msgr2=0x7faee41617c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:49.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.485+0000 7faee2000640 1 --2- 172.21.15.26:0/1488650648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faee4160500 0x7faee41617c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.485+0000 7faee2000640 1 -- 172.21.15.26:0/1488650648 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee40cef40 con 0x7faee415f680 2024-04-26T21:08:49.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.486+0000 7faee2000640 1 --2- 172.21.15.26:0/1488650648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faee415f680 0x7faee40ce5d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faed000e990 tx=0x7faed000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:49.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.486+0000 7faedb400640 1 -- 172.21.15.26:0/1488650648 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0017070 con 0x7faee415f680 2024-04-26T21:08:49.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.486+0000 7faedb400640 1 -- 172.21.15.26:0/1488650648 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed000c9e0 con 0x7faee415f680 2024-04-26T21:08:49.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.486+0000 7faedb400640 1 -- 172.21.15.26:0/1488650648 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed000cdb0 con 0x7faee415f680 2024-04-26T21:08:49.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.486+0000 7faee9e00640 1 -- 172.21.15.26:0/1488650648 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee40cc4d0 con 0x7faee415f680 2024-04-26T21:08:49.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.486+0000 7faee9e00640 1 -- 172.21.15.26:0/1488650648 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee4159aa0 con 0x7faee415f680 2024-04-26T21:08:49.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.487+0000 7faee9e00640 1 -- 172.21.15.26:0/1488650648 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee406a700 con 0x7faee415f680 2024-04-26T21:08:49.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.488+0000 7faedb400640 1 -- 172.21.15.26:0/1488650648 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faed001a070 con 0x7faee415f680 2024-04-26T21:08:49.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.489+0000 7faedb400640 1 --2- 172.21.15.26:0/1488650648 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7faec406c8f0 0x7faec406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:49.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.489+0000 7faedb400640 1 -- 172.21.15.26:0/1488650648 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7faed008bcf0 con 0x7faee415f680 2024-04-26T21:08:49.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.489+0000 7faee2a00640 1 --2- 172.21.15.26:0/1488650648 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7faec406c8f0 0x7faec406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:49.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.490+0000 7faee2a00640 1 --2- 172.21.15.26:0/1488650648 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7faec406c8f0 0x7faec406edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faed4034000 tx=0x7faed402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:49.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.494+0000 7faedb400640 1 -- 172.21.15.26:0/1488650648 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faed0059ac0 con 0x7faee415f680 2024-04-26T21:08:49.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.698+0000 7faee9e00640 1 -- 172.21.15.26:0/1488650648 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faee40d2760 con 0x7faee415f680 2024-04-26T21:08:49.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.701+0000 7faedb400640 1 -- 172.21.15.26:0/1488650648 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16557 (secure 0 0 0) 0x7faed005ddc0 con 0x7faee415f680 2024-04-26T21:08:49.702 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:49.703 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:08:39.177854+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:08:35.481169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:08:36.609498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:08:49.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.703+0000 7faee9e00640 1 -- 172.21.15.26:0/1488650648 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7faec406c8f0 msgr2=0x7faec406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:49.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.703+0000 7faee9e00640 1 --2- 172.21.15.26:0/1488650648 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7faec406c8f0 0x7faec406edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faed4034000 tx=0x7faed402e000 comp rx=0 tx=0).stop 2024-04-26T21:08:49.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.703+0000 7faee9e00640 1 -- 172.21.15.26:0/1488650648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faee415f680 msgr2=0x7faee40ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:49.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.703+0000 7faee9e00640 1 --2- 172.21.15.26:0/1488650648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faee415f680 0x7faee40ce5d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faed000e990 tx=0x7faed000ee50 comp rx=0 tx=0).stop 2024-04-26T21:08:49.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.703+0000 7faee9e00640 1 -- 172.21.15.26:0/1488650648 shutdown_connections 2024-04-26T21:08:49.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.703+0000 7faee9e00640 1 --2- 172.21.15.26:0/1488650648 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7faec406c8f0 0x7faec406edb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.703+0000 7faee9e00640 1 --2- 172.21.15.26:0/1488650648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faee4160500 0x7faee41617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.703+0000 7faee9e00640 1 --2- 172.21.15.26:0/1488650648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faee415f680 0x7faee40ce5d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.703+0000 7faee9e00640 1 --2- 172.21.15.26:0/1488650648 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faee40d1b80 0x7faee4159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.703+0000 7faee9e00640 1 -- 172.21.15.26:0/1488650648 >> 172.21.15.26:0/1488650648 conn(0x7faee409e490 msgr2=0x7faee4160ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.704+0000 7faee9e00640 1 -- 172.21.15.26:0/1488650648 shutdown_connections 2024-04-26T21:08:49.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.704+0000 7faee9e00640 1 -- 172.21.15.26:0/1488650648 wait complete. 2024-04-26T21:08:49.721 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:08:31.322511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:08:35.481169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:08:36.609498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:08:49.722 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:08:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.927+0000 7fe679a00640 1 Processor -- start 2024-04-26T21:08:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.928+0000 7fe679a00640 1 -- start start 2024-04-26T21:08:49.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.929+0000 7fe679a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe674160a80 0x7fe6740b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:49.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.929+0000 7fe679a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe674072d90 con 0x7fe674160e60 2024-04-26T21:08:49.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.929+0000 7fe679a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe674073070 con 0x7fe674072490 2024-04-26T21:08:49.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.929+0000 7fe679a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fe6740b5be0 con 0x7fe674160a80 2024-04-26T21:08:49.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.930+0000 7fe673e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe674160a80 0x7fe6740b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:49.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.930+0000 7fe673e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe674160a80 0x7fe6740b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34776/0 (socket says 172.21.15.26:34776) 2024-04-26T21:08:49.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.930+0000 7fe673e00640 1 -- 172.21.15.26:0/2494758549 learned_addr learned my addr 172.21.15.26:0/2494758549 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:49.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.930+0000 7fe673e00640 1 -- 172.21.15.26:0/2494758549 >> v1:172.21.15.86:6789/0 conn(0x7fe674072490 legacy=0x7fe6740b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:49.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.930+0000 7fe673e00640 1 -- 172.21.15.26:0/2494758549 >> v1:172.21.15.26:6789/0 conn(0x7fe674160e60 legacy=0x7fe67406e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:49.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.930+0000 7fe673e00640 1 -- 172.21.15.26:0/2494758549 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe674073350 con 0x7fe674160a80 2024-04-26T21:08:49.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.931+0000 7fe673e00640 1 --2- 172.21.15.26:0/2494758549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe674160a80 0x7fe6740b4040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe664009870 tx=0x7fe66402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f688bd093d9019 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:49.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.931+0000 7fe672a00640 1 -- 172.21.15.26:0/2494758549 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664037070 con 0x7fe674160a80 2024-04-26T21:08:49.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.931+0000 7fe672a00640 1 -- 172.21.15.26:0/2494758549 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe664032440 con 0x7fe674160a80 2024-04-26T21:08:49.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.931+0000 7fe672a00640 1 -- 172.21.15.26:0/2494758549 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664032810 con 0x7fe674160a80 2024-04-26T21:08:49.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.932+0000 7fe679a00640 1 -- 172.21.15.26:0/2494758549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe674160a80 msgr2=0x7fe6740b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:49.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.932+0000 7fe679a00640 1 --2- 172.21.15.26:0/2494758549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe674160a80 0x7fe6740b4040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe664009870 tx=0x7fe66402a2a0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.932+0000 7fe679a00640 1 -- 172.21.15.26:0/2494758549 shutdown_connections 2024-04-26T21:08:49.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.932+0000 7fe679a00640 1 --2- 172.21.15.26:0/2494758549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe674160a80 0x7fe6740b4040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.932+0000 7fe679a00640 1 -- 172.21.15.26:0/2494758549 >> 172.21.15.26:0/2494758549 conn(0x7fe6740b2a10 msgr2=0x7fe6740b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:49.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.932+0000 7fe679a00640 1 -- 172.21.15.26:0/2494758549 shutdown_connections 2024-04-26T21:08:49.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.932+0000 7fe679a00640 1 -- 172.21.15.26:0/2494758549 wait complete. 2024-04-26T21:08:49.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.934+0000 7fe679a00640 1 Processor -- start 2024-04-26T21:08:49.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.934+0000 7fe679a00640 1 -- start start 2024-04-26T21:08:49.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.935+0000 7fe679a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe674072490 0x7fe674140930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:49.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.935+0000 7fe679a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe674160a80 0x7fe67406e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:49.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.935+0000 7fe679a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe674160e60 0x7fe67406e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:49.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.935+0000 7fe679a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe674072900 con 0x7fe674072490 2024-04-26T21:08:49.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.935+0000 7fe679a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6741612d0 con 0x7fe674160a80 2024-04-26T21:08:49.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.935+0000 7fe679a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe674072a80 con 0x7fe674160e60 2024-04-26T21:08:49.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.935+0000 7fe673e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe674072490 0x7fe674140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:49.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.935+0000 7fe673e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe674072490 0x7fe674140930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34784/0 (socket says 172.21.15.26:34784) 2024-04-26T21:08:49.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.935+0000 7fe673e00640 1 -- 172.21.15.26:0/319319486 learned_addr learned my addr 172.21.15.26:0/319319486 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:49.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.936+0000 7fe673e00640 1 -- 172.21.15.26:0/319319486 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe674160e60 msgr2=0x7fe67406e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:49.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.936+0000 7fe673e00640 1 --2- 172.21.15.26:0/319319486 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe674160e60 0x7fe67406e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.936+0000 7fe673e00640 1 -- 172.21.15.26:0/319319486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe674160a80 msgr2=0x7fe67406e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:49.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.936+0000 7fe673e00640 1 --2- 172.21.15.26:0/319319486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe674160a80 0x7fe67406e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:49.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.936+0000 7fe673e00640 1 -- 172.21.15.26:0/319319486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6741410b0 con 0x7fe674072490 2024-04-26T21:08:49.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.936+0000 7fe673e00640 1 --2- 172.21.15.26:0/319319486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe674072490 0x7fe674140930 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe664009870 tx=0x7fe6640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:49.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.936+0000 7fe66be00640 1 -- 172.21.15.26:0/319319486 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664037040 con 0x7fe674072490 2024-04-26T21:08:49.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.936+0000 7fe66be00640 1 -- 172.21.15.26:0/319319486 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe664042070 con 0x7fe674072490 2024-04-26T21:08:49.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.936+0000 7fe66be00640 1 -- 172.21.15.26:0/319319486 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664030bb0 con 0x7fe674072490 2024-04-26T21:08:49.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.936+0000 7fe679a00640 1 -- 172.21.15.26:0/319319486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6741f5f80 con 0x7fe674072490 2024-04-26T21:08:49.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.936+0000 7fe679a00640 1 -- 172.21.15.26:0/319319486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe674141630 con 0x7fe674072490 2024-04-26T21:08:49.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.938+0000 7fe679a00640 1 -- 172.21.15.26:0/319319486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe67406a700 con 0x7fe674072490 2024-04-26T21:08:49.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.938+0000 7fe66be00640 1 -- 172.21.15.26:0/319319486 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe664047460 con 0x7fe674072490 2024-04-26T21:08:49.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.939+0000 7fe66be00640 1 --2- 172.21.15.26:0/319319486 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe65406c940 0x7fe65406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:49.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.939+0000 7fe66be00640 1 -- 172.21.15.26:0/319319486 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe6640ac790 con 0x7fe674072490 2024-04-26T21:08:49.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.939+0000 7fe673400640 1 --2- 172.21.15.26:0/319319486 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe65406c940 0x7fe65406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:49.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.940+0000 7fe673400640 1 --2- 172.21.15.26:0/319319486 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe65406c940 0x7fe65406ee00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe660000f60 tx=0x7fe6600079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:49.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:49.944+0000 7fe66be00640 1 -- 172.21.15.26:0/319319486 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe664045360 con 0x7fe674072490 2024-04-26T21:08:50.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.147+0000 7fe679a00640 1 -- 172.21.15.26:0/319319486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe674160840 con 0x7fe674072490 2024-04-26T21:08:50.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.147+0000 7fe66be00640 1 -- 172.21.15.26:0/319319486 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fe66407aba0 con 0x7fe674072490 2024-04-26T21:08:50.148 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-26T21:08:50.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.150+0000 7fe679a00640 1 -- 172.21.15.26:0/319319486 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe65406c940 msgr2=0x7fe65406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:50.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.150+0000 7fe679a00640 1 --2- 172.21.15.26:0/319319486 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe65406c940 0x7fe65406ee00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe660000f60 tx=0x7fe6600079d0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.150+0000 7fe679a00640 1 -- 172.21.15.26:0/319319486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe674072490 msgr2=0x7fe674140930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:50.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.150+0000 7fe679a00640 1 --2- 172.21.15.26:0/319319486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe674072490 0x7fe674140930 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe664009870 tx=0x7fe6640046c0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.150+0000 7fe679a00640 1 -- 172.21.15.26:0/319319486 shutdown_connections 2024-04-26T21:08:50.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.150+0000 7fe679a00640 1 --2- 172.21.15.26:0/319319486 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe65406c940 0x7fe65406ee00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.150+0000 7fe679a00640 1 --2- 172.21.15.26:0/319319486 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe674160e60 0x7fe67406e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.150+0000 7fe679a00640 1 --2- 172.21.15.26:0/319319486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe674160a80 0x7fe67406e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.150+0000 7fe679a00640 1 --2- 172.21.15.26:0/319319486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe674072490 0x7fe674140930 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.150+0000 7fe679a00640 1 -- 172.21.15.26:0/319319486 >> 172.21.15.26:0/319319486 conn(0x7fe6740b2a10 msgr2=0x7fe6740b3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:50.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.150+0000 7fe679a00640 1 -- 172.21.15.26:0/319319486 shutdown_connections 2024-04-26T21:08:50.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.151+0000 7fe679a00640 1 -- 172.21.15.26:0/319319486 wait complete. 2024-04-26T21:08:50.171 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:08:50.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.385+0000 7ff9fcc00640 1 Processor -- start 2024-04-26T21:08:50.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.385+0000 7ff9fcc00640 1 -- start start 2024-04-26T21:08:50.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.387+0000 7ff9fcc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff9f8160a80 0x7ff9f80b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.387+0000 7ff9fcc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff9f8160e60 0x7ff9f806e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.387+0000 7ff9fcc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9f8072490 0x7ff9f80b6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.387+0000 7ff9fcc00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ff9f80b5be0 con 0x7ff9f8160e60 2024-04-26T21:08:50.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.387+0000 7ff9fcc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff9f8072d90 con 0x7ff9f8160a80 2024-04-26T21:08:50.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.387+0000 7ff9fcc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f8073070 con 0x7ff9f8072490 2024-04-26T21:08:50.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.387+0000 7ff9f6a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff9f8160e60 0x7ff9f806e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:50.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.387+0000 7ff9f6a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff9f8160e60 0x7ff9f806e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34794/0 (socket says 172.21.15.26:34794) 2024-04-26T21:08:50.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.387+0000 7ff9f6a00640 1 -- 172.21.15.26:0/1547490156 learned_addr learned my addr 172.21.15.26:0/1547490156 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:50.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.388+0000 7ff9f6a00640 1 -- 172.21.15.26:0/1547490156 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9f8072490 msgr2=0x7ff9f80b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:50.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.388+0000 7ff9f7e00640 1 --2- 172.21.15.26:0/1547490156 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9f8072490 0x7ff9f80b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:50.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.388+0000 7ff9f6a00640 1 --2- 172.21.15.26:0/1547490156 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9f8072490 0x7ff9f80b6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.388+0000 7ff9f7400640 1 --2- 172.21.15.26:0/1547490156 >> v2:172.21.15.86:3300/0 conn(0x7ff9f8160a80 0x7ff9f80b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:50.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.388+0000 7ff9f6a00640 1 -- 172.21.15.26:0/1547490156 >> v2:172.21.15.86:3300/0 conn(0x7ff9f8160a80 msgr2=0x7ff9f80b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:50.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.388+0000 7ff9f6a00640 1 --2- 172.21.15.26:0/1547490156 >> v2:172.21.15.86:3300/0 conn(0x7ff9f8160a80 0x7ff9f80b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.388+0000 7ff9f6a00640 1 -- 172.21.15.26:0/1547490156 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f8073350 con 0x7ff9f8160e60 2024-04-26T21:08:50.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.389+0000 7ff9f6a00640 1 --2- 172.21.15.26:0/1547490156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9f8160e60 0x7ff9f806e210 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8051090 tx=0x7ff9e402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53abc1c993d17d2b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:50.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.389+0000 7ff9f6000640 1 -- 172.21.15.26:0/1547490156 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4037070 con 0x7ff9f8160e60 2024-04-26T21:08:50.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.389+0000 7ff9f6000640 1 -- 172.21.15.26:0/1547490156 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9e4032440 con 0x7ff9f8160e60 2024-04-26T21:08:50.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.389+0000 7ff9f6000640 1 -- 172.21.15.26:0/1547490156 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4032810 con 0x7ff9f8160e60 2024-04-26T21:08:50.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.390+0000 7ff9fcc00640 1 -- 172.21.15.26:0/1547490156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9f8160e60 msgr2=0x7ff9f806e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:50.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.390+0000 7ff9fcc00640 1 --2- 172.21.15.26:0/1547490156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9f8160e60 0x7ff9f806e210 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8051090 tx=0x7ff9e402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.390+0000 7ff9fcc00640 1 -- 172.21.15.26:0/1547490156 shutdown_connections 2024-04-26T21:08:50.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.390+0000 7ff9fcc00640 1 --2- 172.21.15.26:0/1547490156 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9f8072490 0x7ff9f80b6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.390+0000 7ff9fcc00640 1 --2- 172.21.15.26:0/1547490156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9f8160e60 0x7ff9f806e210 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.390+0000 7ff9fcc00640 1 --2- 172.21.15.26:0/1547490156 >> v2:172.21.15.86:3300/0 conn(0x7ff9f8160a80 0x7ff9f80b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.390+0000 7ff9fcc00640 1 -- 172.21.15.26:0/1547490156 >> 172.21.15.26:0/1547490156 conn(0x7ff9f80b2a10 msgr2=0x7ff9f80b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:50.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.390+0000 7ff9fcc00640 1 -- 172.21.15.26:0/1547490156 shutdown_connections 2024-04-26T21:08:50.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.390+0000 7ff9fcc00640 1 -- 172.21.15.26:0/1547490156 wait complete. 2024-04-26T21:08:50.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.392+0000 7ff9fcc00640 1 Processor -- start 2024-04-26T21:08:50.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.392+0000 7ff9fcc00640 1 -- start start 2024-04-26T21:08:50.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.393+0000 7ff9fcc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9f8072490 0x7ff9f814c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.393+0000 7ff9fcc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9f8160a80 0x7ff9f81487d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.393+0000 7ff9fcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9f8160e60 0x7ff9f80af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.393+0000 7ff9fcc00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f81612d0 con 0x7ff9f8072490 2024-04-26T21:08:50.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.393+0000 7ff9fcc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f80b5820 con 0x7ff9f8160e60 2024-04-26T21:08:50.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.393+0000 7ff9fcc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f8072920 con 0x7ff9f8160a80 2024-04-26T21:08:50.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.393+0000 7ff9f7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9f8072490 0x7ff9f814c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:50.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.393+0000 7ff9f7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9f8072490 0x7ff9f814c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34804/0 (socket says 172.21.15.26:34804) 2024-04-26T21:08:50.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.393+0000 7ff9f7400640 1 -- 172.21.15.26:0/2924806182 learned_addr learned my addr 172.21.15.26:0/2924806182 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:50.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.393+0000 7ff9f6a00640 1 --2- 172.21.15.26:0/2924806182 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9f8160a80 0x7ff9f81487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:50.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.393+0000 7ff9f7400640 1 -- 172.21.15.26:0/2924806182 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9f8160a80 msgr2=0x7ff9f81487d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:50.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.393+0000 7ff9f7400640 1 --2- 172.21.15.26:0/2924806182 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9f8160a80 0x7ff9f81487d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.393+0000 7ff9f7400640 1 -- 172.21.15.26:0/2924806182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9f8160e60 msgr2=0x7ff9f80af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:50.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.393+0000 7ff9f7400640 1 --2- 172.21.15.26:0/2924806182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9f8160e60 0x7ff9f80af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.393+0000 7ff9f7400640 1 -- 172.21.15.26:0/2924806182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f80cf5b0 con 0x7ff9f8072490 2024-04-26T21:08:50.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.394+0000 7ff9f7400640 1 --2- 172.21.15.26:0/2924806182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9f8072490 0x7ff9f814c4f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff9e800c7f0 tx=0x7ff9e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:50.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.394+0000 7ff9ef400640 1 -- 172.21.15.26:0/2924806182 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8015070 con 0x7ff9f8072490 2024-04-26T21:08:50.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.394+0000 7ff9ef400640 1 -- 172.21.15.26:0/2924806182 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9e8004240 con 0x7ff9f8072490 2024-04-26T21:08:50.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.394+0000 7ff9ef400640 1 -- 172.21.15.26:0/2924806182 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8007620 con 0x7ff9f8072490 2024-04-26T21:08:50.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.394+0000 7ff9fcc00640 1 -- 172.21.15.26:0/2924806182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f80cf890 con 0x7ff9f8072490 2024-04-26T21:08:50.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.394+0000 7ff9fcc00640 1 -- 172.21.15.26:0/2924806182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f815b3a0 con 0x7ff9f8072490 2024-04-26T21:08:50.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.395+0000 7ff9fcc00640 1 -- 172.21.15.26:0/2924806182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9f806a700 con 0x7ff9f8072490 2024-04-26T21:08:50.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.396+0000 7ff9ef400640 1 -- 172.21.15.26:0/2924806182 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9e80079d0 con 0x7ff9f8072490 2024-04-26T21:08:50.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.397+0000 7ff9ef400640 1 --2- 172.21.15.26:0/2924806182 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff9d806c8f0 0x7ff9d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.397+0000 7ff9ef400640 1 -- 172.21.15.26:0/2924806182 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff9e808bf70 con 0x7ff9f8072490 2024-04-26T21:08:50.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.397+0000 7ff9f6a00640 1 --2- 172.21.15.26:0/2924806182 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff9d806c8f0 0x7ff9d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:50.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.398+0000 7ff9f6a00640 1 --2- 172.21.15.26:0/2924806182 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff9d806c8f0 0x7ff9d806edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4034000 tx=0x7ff9e402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:50.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.401+0000 7ff9ef400640 1 -- 172.21.15.26:0/2924806182 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff9e805a390 con 0x7ff9f8072490 2024-04-26T21:08:50.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.607+0000 7ff9fcc00640 1 -- 172.21.15.26:0/2924806182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff9f8072270 con 0x7ff9f8072490 2024-04-26T21:08:50.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.608+0000 7ff9ef400640 1 -- 172.21.15.26:0/2924806182 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7ff9e8059c90 con 0x7ff9f8072490 2024-04-26T21:08:50.609 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-26T21:08:50.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.611+0000 7ff9fcc00640 1 -- 172.21.15.26:0/2924806182 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff9d806c8f0 msgr2=0x7ff9d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:50.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.611+0000 7ff9fcc00640 1 --2- 172.21.15.26:0/2924806182 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff9d806c8f0 0x7ff9d806edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4034000 tx=0x7ff9e402e000 comp rx=0 tx=0).stop 2024-04-26T21:08:50.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.611+0000 7ff9fcc00640 1 -- 172.21.15.26:0/2924806182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9f8072490 msgr2=0x7ff9f814c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:50.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.611+0000 7ff9fcc00640 1 --2- 172.21.15.26:0/2924806182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9f8072490 0x7ff9f814c4f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff9e800c7f0 tx=0x7ff9e800ccb0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.612+0000 7ff9fcc00640 1 -- 172.21.15.26:0/2924806182 shutdown_connections 2024-04-26T21:08:50.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.612+0000 7ff9fcc00640 1 --2- 172.21.15.26:0/2924806182 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff9d806c8f0 0x7ff9d806edb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.612+0000 7ff9fcc00640 1 --2- 172.21.15.26:0/2924806182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9f8160e60 0x7ff9f80af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.612+0000 7ff9fcc00640 1 --2- 172.21.15.26:0/2924806182 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9f8160a80 0x7ff9f81487d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.612+0000 7ff9fcc00640 1 --2- 172.21.15.26:0/2924806182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9f8072490 0x7ff9f814c4f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.612+0000 7ff9fcc00640 1 -- 172.21.15.26:0/2924806182 >> 172.21.15.26:0/2924806182 conn(0x7ff9f80b2a10 msgr2=0x7ff9f815e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:50.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.612+0000 7ff9fcc00640 1 -- 172.21.15.26:0/2924806182 shutdown_connections 2024-04-26T21:08:50.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.612+0000 7ff9fcc00640 1 -- 172.21.15.26:0/2924806182 wait complete. 2024-04-26T21:08:50.633 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:08:50.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.851+0000 7f4245800640 1 Processor -- start 2024-04-26T21:08:50.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.852+0000 7f4245800640 1 -- start start 2024-04-26T21:08:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.853+0000 7f4245800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f42400b74d0 0x7f42400d0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.853+0000 7f4245800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f42400d0ab0 0x7f42400d3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.853+0000 7f4245800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42400d39c0 0x7f424015eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.853+0000 7f4245800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f42400d3e30 con 0x7f42400d0ab0 2024-04-26T21:08:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.853+0000 7f4245800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f42400b7f90 con 0x7f42400b74d0 2024-04-26T21:08:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.853+0000 7f4245800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f42400b8270 con 0x7f42400d39c0 2024-04-26T21:08:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.854+0000 7f423f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f42400d0ab0 0x7f42400d3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.854+0000 7f423f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f42400d0ab0 0x7f42400d3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34806/0 (socket says 172.21.15.26:34806) 2024-04-26T21:08:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.854+0000 7f423f400640 1 -- 172.21.15.26:0/941977601 learned_addr learned my addr 172.21.15.26:0/941977601 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.854+0000 7f4244e00640 1 --2- 172.21.15.26:0/941977601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42400d39c0 0x7f424015eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.854+0000 7f423f400640 1 -- 172.21.15.26:0/941977601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42400d39c0 msgr2=0x7f424015eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.854+0000 7f423fe00640 1 --2- 172.21.15.26:0/941977601 >> v2:172.21.15.86:3300/0 conn(0x7f42400b74d0 0x7f42400d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:50.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.854+0000 7f423f400640 1 --2- 172.21.15.26:0/941977601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42400d39c0 0x7f424015eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.854+0000 7f423f400640 1 -- 172.21.15.26:0/941977601 >> v2:172.21.15.86:3300/0 conn(0x7f42400b74d0 msgr2=0x7f42400d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.854+0000 7f423f400640 1 --2- 172.21.15.26:0/941977601 >> v2:172.21.15.86:3300/0 conn(0x7f42400b74d0 0x7f42400d0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.854+0000 7f423f400640 1 -- 172.21.15.26:0/941977601 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42400b8550 con 0x7f42400d0ab0 2024-04-26T21:08:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.855+0000 7f423f400640 1 --2- 172.21.15.26:0/941977601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42400d0ab0 0x7f42400d3480 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f424008f4f0 tx=0x7f422802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d13e42355ad077 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.856+0000 7f423ea00640 1 -- 172.21.15.26:0/941977601 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4228037070 con 0x7f42400d0ab0 2024-04-26T21:08:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.856+0000 7f423ea00640 1 -- 172.21.15.26:0/941977601 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4228032440 con 0x7f42400d0ab0 2024-04-26T21:08:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.856+0000 7f423ea00640 1 -- 172.21.15.26:0/941977601 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4228032810 con 0x7f42400d0ab0 2024-04-26T21:08:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.856+0000 7f4245800640 1 -- 172.21.15.26:0/941977601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42400d0ab0 msgr2=0x7f42400d3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.856+0000 7f4245800640 1 --2- 172.21.15.26:0/941977601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42400d0ab0 0x7f42400d3480 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f424008f4f0 tx=0x7f422802a1e0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.856+0000 7f4245800640 1 -- 172.21.15.26:0/941977601 shutdown_connections 2024-04-26T21:08:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.856+0000 7f4245800640 1 --2- 172.21.15.26:0/941977601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42400d39c0 0x7f424015eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.856+0000 7f4245800640 1 --2- 172.21.15.26:0/941977601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42400d0ab0 0x7f42400d3480 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.856+0000 7f4245800640 1 --2- 172.21.15.26:0/941977601 >> v2:172.21.15.86:3300/0 conn(0x7f42400b74d0 0x7f42400d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.856+0000 7f4245800640 1 -- 172.21.15.26:0/941977601 >> 172.21.15.26:0/941977601 conn(0x7f42400d4f30 msgr2=0x7f42400d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:50.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.856+0000 7f4245800640 1 -- 172.21.15.26:0/941977601 shutdown_connections 2024-04-26T21:08:50.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.857+0000 7f4245800640 1 -- 172.21.15.26:0/941977601 wait complete. 2024-04-26T21:08:50.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.858+0000 7f4245800640 1 Processor -- start 2024-04-26T21:08:50.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.858+0000 7f4245800640 1 -- start start 2024-04-26T21:08:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.859+0000 7f4245800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42400b74d0 0x7f42400a27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.859+0000 7f4245800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42400d0ab0 0x7f424015e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.859+0000 7f4245800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42400d39c0 0x7f42401e87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.859+0000 7f4245800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f42400b78b0 con 0x7f42400d39c0 2024-04-26T21:08:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.859+0000 7f4245800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42400b8b10 con 0x7f42400d0ab0 2024-04-26T21:08:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.860+0000 7f4245800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f424015f0c0 con 0x7f42400b74d0 2024-04-26T21:08:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.860+0000 7f4244e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42400d39c0 0x7f42401e87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.860+0000 7f4244e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42400d39c0 0x7f42401e87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34816/0 (socket says 172.21.15.26:34816) 2024-04-26T21:08:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.860+0000 7f4244e00640 1 -- 172.21.15.26:0/2561730689 learned_addr learned my addr 172.21.15.26:0/2561730689 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.860+0000 7f4244e00640 1 -- 172.21.15.26:0/2561730689 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42400b74d0 msgr2=0x7f42400a27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.860+0000 7f423fe00640 1 --2- 172.21.15.26:0/2561730689 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42400b74d0 0x7f42400a27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.860+0000 7f4244e00640 1 --2- 172.21.15.26:0/2561730689 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42400b74d0 0x7f42400a27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.860+0000 7f4244e00640 1 -- 172.21.15.26:0/2561730689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42400d0ab0 msgr2=0x7f424015e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:50.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.860+0000 7f4244e00640 1 --2- 172.21.15.26:0/2561730689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42400d0ab0 0x7f424015e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:50.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.860+0000 7f4244e00640 1 -- 172.21.15.26:0/2561730689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42401e8f70 con 0x7f42400d39c0 2024-04-26T21:08:50.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.860+0000 7f4244e00640 1 --2- 172.21.15.26:0/2561730689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42400d39c0 0x7f42401e87d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f423800d600 tx=0x7f423800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:50.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.861+0000 7f4237e00640 1 -- 172.21.15.26:0/2561730689 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42380125f0 con 0x7f42400d39c0 2024-04-26T21:08:50.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.861+0000 7f4237e00640 1 -- 172.21.15.26:0/2561730689 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4238019070 con 0x7f42400d39c0 2024-04-26T21:08:50.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.861+0000 7f4237e00640 1 -- 172.21.15.26:0/2561730689 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42380129f0 con 0x7f42400d39c0 2024-04-26T21:08:50.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.861+0000 7f4245800640 1 -- 172.21.15.26:0/2561730689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42401e9250 con 0x7f42400d39c0 2024-04-26T21:08:50.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.861+0000 7f4245800640 1 -- 172.21.15.26:0/2561730689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f424009ef70 con 0x7f42400d39c0 2024-04-26T21:08:50.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.862+0000 7f4245800640 1 -- 172.21.15.26:0/2561730689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42400506c0 con 0x7f42400d39c0 2024-04-26T21:08:50.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.863+0000 7f4237e00640 1 -- 172.21.15.26:0/2561730689 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4238012b90 con 0x7f42400d39c0 2024-04-26T21:08:50.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.864+0000 7f4237e00640 1 --2- 172.21.15.26:0/2561730689 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f421c06c920 0x7f421c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:50.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.864+0000 7f4237e00640 1 -- 172.21.15.26:0/2561730689 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f423808bf80 con 0x7f42400d39c0 2024-04-26T21:08:50.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.864+0000 7f423fe00640 1 --2- 172.21.15.26:0/2561730689 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f421c06c920 0x7f421c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:50.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.865+0000 7f423fe00640 1 --2- 172.21.15.26:0/2561730689 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f421c06c920 0x7f421c06ede0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4230008e70 tx=0x7f42300083c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:50.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:50.869+0000 7f4237e00640 1 -- 172.21.15.26:0/2561730689 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f423805a320 con 0x7f42400d39c0 2024-04-26T21:08:51.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.071+0000 7f4245800640 1 -- 172.21.15.26:0/2561730689 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f42400d5d40 con 0x7f42400d39c0 2024-04-26T21:08:51.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.072+0000 7f4237e00640 1 -- 172.21.15.26:0/2561730689 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f4238059c20 con 0x7f42400d39c0 2024-04-26T21:08:51.073 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-26T21:08:51.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.075+0000 7f4245800640 1 -- 172.21.15.26:0/2561730689 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f421c06c920 msgr2=0x7f421c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:51.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.075+0000 7f4245800640 1 --2- 172.21.15.26:0/2561730689 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f421c06c920 0x7f421c06ede0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4230008e70 tx=0x7f42300083c0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.075+0000 7f4245800640 1 -- 172.21.15.26:0/2561730689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42400d39c0 msgr2=0x7f42401e87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:51.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.075+0000 7f4245800640 1 --2- 172.21.15.26:0/2561730689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42400d39c0 0x7f42401e87d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f423800d600 tx=0x7f423800dac0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.075+0000 7f4245800640 1 -- 172.21.15.26:0/2561730689 shutdown_connections 2024-04-26T21:08:51.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.075+0000 7f4245800640 1 --2- 172.21.15.26:0/2561730689 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f421c06c920 0x7f421c06ede0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.075+0000 7f4245800640 1 --2- 172.21.15.26:0/2561730689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42400d39c0 0x7f42401e87d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.075+0000 7f4245800640 1 --2- 172.21.15.26:0/2561730689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42400d0ab0 0x7f424015e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.075+0000 7f4245800640 1 --2- 172.21.15.26:0/2561730689 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42400b74d0 0x7f42400a27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.075+0000 7f4245800640 1 -- 172.21.15.26:0/2561730689 >> 172.21.15.26:0/2561730689 conn(0x7f42400d4f30 msgr2=0x7f424006e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:51.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.075+0000 7f4245800640 1 -- 172.21.15.26:0/2561730689 shutdown_connections 2024-04-26T21:08:51.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.075+0000 7f4245800640 1 -- 172.21.15.26:0/2561730689 wait complete. 2024-04-26T21:08:51.094 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:08:51.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.291+0000 7f2d0a000640 1 Processor -- start 2024-04-26T21:08:51.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.291+0000 7f2d0a000640 1 -- start start 2024-04-26T21:08:51.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.293+0000 7f2d0a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d0406f340 0x7f2d0415faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:51.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.293+0000 7f2d0a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2d0415ffe0 0x7f2d040d11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:51.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.293+0000 7f2d0a000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d0406f7d0 con 0x7f2d04073470 2024-04-26T21:08:51.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.293+0000 7f2d0a000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2d04073f60 con 0x7f2d0415ffe0 2024-04-26T21:08:51.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.293+0000 7f2d0a000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2d0406fab0 con 0x7f2d0406f340 2024-04-26T21:08:51.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.293+0000 7f2d08c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2d04073470 0x7f2d04073850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55162/0 (socket says 172.21.15.26:55162) 2024-04-26T21:08:51.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.293+0000 7f2d08c00640 1 -- 172.21.15.26:0/821582328 learned_addr learned my addr 172.21.15.26:0/821582328 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:51.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.294+0000 7f2cfbe00640 1 --2- 172.21.15.26:0/821582328 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d0406f340 0x7f2d0415faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:51.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.294+0000 7f2d09600640 1 --2- 172.21.15.26:0/821582328 >> v2:172.21.15.86:3300/0 conn(0x7f2d0415ffe0 0x7f2d040d11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:51.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.294+0000 7f2d03e00640 1 -- 172.21.15.26:0/821582328 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2565456735 0 0) 0x7f2d0406f7d0 con 0x7f2d04073470 2024-04-26T21:08:51.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.294+0000 7f2d03e00640 1 -- 172.21.15.26:0/821582328 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ce8005730 con 0x7f2d04073470 2024-04-26T21:08:51.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.294+0000 7f2d09600640 1 -- 172.21.15.26:0/821582328 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d0406f340 msgr2=0x7f2d0415faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:51.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.294+0000 7f2d09600640 1 --2- 172.21.15.26:0/821582328 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d0406f340 0x7f2d0415faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.294+0000 7f2d09600640 1 -- 172.21.15.26:0/821582328 >> v1:172.21.15.26:6789/0 conn(0x7f2d04073470 legacy=0x7f2d04073850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:51.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.294+0000 7f2d09600640 1 -- 172.21.15.26:0/821582328 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0406fd90 con 0x7f2d0415ffe0 2024-04-26T21:08:51.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.294+0000 7f2cfbe00640 1 --2- 172.21.15.26:0/821582328 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d0406f340 0x7f2d0415faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:08:51.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.296+0000 7f2d09600640 1 --2- 172.21.15.26:0/821582328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0415ffe0 0x7f2d040d11f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009d90 tx=0x7f2cfc02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd94374b65271219 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:51.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.296+0000 7f2d03e00640 1 -- 172.21.15.26:0/821582328 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc007740 con 0x7f2d0415ffe0 2024-04-26T21:08:51.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.296+0000 7f2d03e00640 1 -- 172.21.15.26:0/821582328 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cfc0078e0 con 0x7f2d0415ffe0 2024-04-26T21:08:51.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.296+0000 7f2d03e00640 1 -- 172.21.15.26:0/821582328 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc007cb0 con 0x7f2d0415ffe0 2024-04-26T21:08:51.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.297+0000 7f2d0a000640 1 -- 172.21.15.26:0/821582328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0415ffe0 msgr2=0x7f2d040d11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:51.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.297+0000 7f2d0a000640 1 --2- 172.21.15.26:0/821582328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0415ffe0 0x7f2d040d11f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009d90 tx=0x7f2cfc02a660 comp rx=0 tx=0).stop 2024-04-26T21:08:51.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.297+0000 7f2d0a000640 1 -- 172.21.15.26:0/821582328 shutdown_connections 2024-04-26T21:08:51.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.297+0000 7f2d0a000640 1 --2- 172.21.15.26:0/821582328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0415ffe0 0x7f2d040d11f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.297+0000 7f2d0a000640 1 --2- 172.21.15.26:0/821582328 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d0406f340 0x7f2d0415faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.297+0000 7f2d0a000640 1 -- 172.21.15.26:0/821582328 >> 172.21.15.26:0/821582328 conn(0x7f2d0406c690 msgr2=0x7f2d0406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:51.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.297+0000 7f2d0a000640 1 -- 172.21.15.26:0/821582328 shutdown_connections 2024-04-26T21:08:51.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.297+0000 7f2d0a000640 1 -- 172.21.15.26:0/821582328 wait complete. 2024-04-26T21:08:51.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.299+0000 7f2d0a000640 1 Processor -- start 2024-04-26T21:08:51.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.299+0000 7f2d0a000640 1 -- start start 2024-04-26T21:08:51.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.300+0000 7f2d0a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d04073470 0x7f2d040b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:51.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.300+0000 7f2d0a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d0415ffe0 0x7f2d04154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:51.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.300+0000 7f2d0a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0415e920 0x7f2d0415edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:51.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.300+0000 7f2d0a000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2d040b4d40 con 0x7f2d04073470 2024-04-26T21:08:51.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.300+0000 7f2d0a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0415c4b0 con 0x7f2d0415e920 2024-04-26T21:08:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.300+0000 7f2d0a000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f2d040b4ec0 con 0x7f2d0415ffe0 2024-04-26T21:08:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.301+0000 7f2d08c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d04073470 0x7f2d040b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.301+0000 7f2d08c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d04073470 0x7f2d040b26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34824/0 (socket says 172.21.15.26:34824) 2024-04-26T21:08:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.301+0000 7f2d08c00640 1 -- 172.21.15.26:0/744160910 learned_addr learned my addr 172.21.15.26:0/744160910 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.301+0000 7f2cfbe00640 1 --2- 172.21.15.26:0/744160910 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d0415ffe0 0x7f2d04154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.301+0000 7f2d09600640 1 --2- 172.21.15.26:0/744160910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0415e920 0x7f2d0415edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.301+0000 7f2d08c00640 1 -- 172.21.15.26:0/744160910 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d0415ffe0 msgr2=0x7f2d04154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.301+0000 7f2d08c00640 1 --2- 172.21.15.26:0/744160910 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d0415ffe0 0x7f2d04154d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.301+0000 7f2d08c00640 1 -- 172.21.15.26:0/744160910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0415e920 msgr2=0x7f2d0415edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.301+0000 7f2d08c00640 1 --2- 172.21.15.26:0/744160910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0415e920 0x7f2d0415edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.301+0000 7f2d08c00640 1 -- 172.21.15.26:0/744160910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d040b1920 con 0x7f2d04073470 2024-04-26T21:08:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.301+0000 7f2d08c00640 1 --2- 172.21.15.26:0/744160910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d04073470 0x7f2d040b26e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400c460 tx=0x7f2cf400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.302+0000 7f2d01600640 1 -- 172.21.15.26:0/744160910 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4017070 con 0x7f2d04073470 2024-04-26T21:08:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.302+0000 7f2d01600640 1 -- 172.21.15.26:0/744160910 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf4005ce0 con 0x7f2d04073470 2024-04-26T21:08:51.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.302+0000 7f2d01600640 1 -- 172.21.15.26:0/744160910 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4007290 con 0x7f2d04073470 2024-04-26T21:08:51.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.302+0000 7f2d0a000640 1 -- 172.21.15.26:0/744160910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d040b1c00 con 0x7f2d04073470 2024-04-26T21:08:51.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.302+0000 7f2d0a000640 1 -- 172.21.15.26:0/744160910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d040b21c0 con 0x7f2d04073470 2024-04-26T21:08:51.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.303+0000 7f2d0a000640 1 -- 172.21.15.26:0/744160910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d040b23c0 con 0x7f2d04073470 2024-04-26T21:08:51.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.304+0000 7f2d01600640 1 -- 172.21.15.26:0/744160910 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2cf401a070 con 0x7f2d04073470 2024-04-26T21:08:51.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.305+0000 7f2d01600640 1 --2- 172.21.15.26:0/744160910 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f2ce006c8f0 0x7f2ce006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:51.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.306+0000 7f2d01600640 1 -- 172.21.15.26:0/744160910 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2cf408afd0 con 0x7f2d04073470 2024-04-26T21:08:51.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.306+0000 7f2cfbe00640 1 --2- 172.21.15.26:0/744160910 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f2ce006c8f0 0x7f2ce006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:51.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.306+0000 7f2cfbe00640 1 --2- 172.21.15.26:0/744160910 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f2ce006c8f0 0x7f2ce006edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2d040b3a70 tx=0x7f2cec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:51.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.310+0000 7f2d01600640 1 -- 172.21.15.26:0/744160910 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2d040b23c0 con 0x7f2d04073470 2024-04-26T21:08:51.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.553+0000 7f2d0a000640 1 -- 172.21.15.26:0/744160910 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d0406a700 con 0x7f2d04073470 2024-04-26T21:08:51.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.554+0000 7f2d01600640 1 -- 172.21.15.26:0/744160910 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7f2cf4059360 con 0x7f2d04073470 2024-04-26T21:08:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:51.555 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/2695371801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2695371801},{"type":"v1","addr":"172.21.15.26:6836","nonce":2695371801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3349732696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3349732696},{"type":"v1","addr":"172.21.15.26:6837","nonce":3349732696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/2288633870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2288633870},{"type":"v1","addr":"172.21.15.86:6833","nonce":2288633870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:08:37.622844+0000","modified":"2024-04-26T21:08:40.715140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/2125360615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2125360615},{"type":"v1","addr":"172.21.15.86:6835","nonce":2125360615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4261,"qdb_cluster":[4261]},"id":1}]} 2024-04-26T21:08:51.555 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2024-04-26T21:08:51.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.557+0000 7f2d0a000640 1 -- 172.21.15.26:0/744160910 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f2ce006c8f0 msgr2=0x7f2ce006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:51.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.557+0000 7f2d0a000640 1 --2- 172.21.15.26:0/744160910 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f2ce006c8f0 0x7f2ce006edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2d040b3a70 tx=0x7f2cec008040 comp rx=0 tx=0).stop 2024-04-26T21:08:51.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.557+0000 7f2d0a000640 1 -- 172.21.15.26:0/744160910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d04073470 msgr2=0x7f2d040b26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:51.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.557+0000 7f2d0a000640 1 --2- 172.21.15.26:0/744160910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d04073470 0x7f2d040b26e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400c460 tx=0x7f2cf400c920 comp rx=0 tx=0).stop 2024-04-26T21:08:51.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.557+0000 7f2d0a000640 1 -- 172.21.15.26:0/744160910 shutdown_connections 2024-04-26T21:08:51.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.557+0000 7f2d0a000640 1 --2- 172.21.15.26:0/744160910 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f2ce006c8f0 0x7f2ce006edb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.557+0000 7f2d0a000640 1 --2- 172.21.15.26:0/744160910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0415e920 0x7f2d0415edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.557+0000 7f2d0a000640 1 --2- 172.21.15.26:0/744160910 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2d0415ffe0 0x7f2d04154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.557+0000 7f2d0a000640 1 --2- 172.21.15.26:0/744160910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d04073470 0x7f2d040b26e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.557+0000 7f2d0a000640 1 -- 172.21.15.26:0/744160910 >> 172.21.15.26:0/744160910 conn(0x7f2d0406c690 msgr2=0x7f2d040af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:51.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.558+0000 7f2d0a000640 1 -- 172.21.15.26:0/744160910 shutdown_connections 2024-04-26T21:08:51.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.558+0000 7f2d0a000640 1 -- 172.21.15.26:0/744160910 wait complete. 2024-04-26T21:08:51.578 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:08:51.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.784+0000 7f4a39a00640 1 Processor -- start 2024-04-26T21:08:51.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.784+0000 7f4a39a00640 1 -- start start 2024-04-26T21:08:51.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.786+0000 7f4a39a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4a34146990 0x7f4a340b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:51.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.786+0000 7f4a39a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4a3406ec20 0x7f4a340d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:51.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.786+0000 7f4a39a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a340d1210 0x7f4a34072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:51.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.786+0000 7f4a39a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f4a34146d70 con 0x7f4a34146990 2024-04-26T21:08:51.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.786+0000 7f4a39a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4a340d1640 con 0x7f4a3406ec20 2024-04-26T21:08:51.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.786+0000 7f4a39a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4a340b4480 con 0x7f4a340d1210 2024-04-26T21:08:51.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.786+0000 7f4a32a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4a34146990 0x7f4a340b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:51.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.786+0000 7f4a32a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4a34146990 0x7f4a340b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34828/0 (socket says 172.21.15.26:34828) 2024-04-26T21:08:51.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.786+0000 7f4a32a00640 1 -- 172.21.15.26:0/2411426516 learned_addr learned my addr 172.21.15.26:0/2411426516 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:51.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.787+0000 7f4a32a00640 1 -- 172.21.15.26:0/2411426516 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a340d1210 msgr2=0x7f4a34072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:51.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.787+0000 7f4a32a00640 1 --2- 172.21.15.26:0/2411426516 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a340d1210 0x7f4a34072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.787+0000 7f4a32a00640 1 -- 172.21.15.26:0/2411426516 >> v2:172.21.15.86:3300/0 conn(0x7f4a3406ec20 msgr2=0x7f4a340d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:51.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.787+0000 7f4a32a00640 1 --2- 172.21.15.26:0/2411426516 >> v2:172.21.15.86:3300/0 conn(0x7f4a3406ec20 0x7f4a340d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.787+0000 7f4a32a00640 1 -- 172.21.15.26:0/2411426516 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a34073b70 con 0x7f4a34146990 2024-04-26T21:08:51.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.788+0000 7f4a32a00640 1 --2- 172.21.15.26:0/2411426516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a34146990 0x7f4a340b3f40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4a3415b0c0 tx=0x7f4a2402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ce18189be8db17 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:51.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.788+0000 7f4a31600640 1 -- 172.21.15.26:0/2411426516 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a24037070 con 0x7f4a34146990 2024-04-26T21:08:51.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.788+0000 7f4a31600640 1 -- 172.21.15.26:0/2411426516 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a24032440 con 0x7f4a34146990 2024-04-26T21:08:51.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.788+0000 7f4a31600640 1 -- 172.21.15.26:0/2411426516 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a24032810 con 0x7f4a34146990 2024-04-26T21:08:51.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.789+0000 7f4a39a00640 1 -- 172.21.15.26:0/2411426516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a34146990 msgr2=0x7f4a340b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:51.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.789+0000 7f4a39a00640 1 --2- 172.21.15.26:0/2411426516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a34146990 0x7f4a340b3f40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4a3415b0c0 tx=0x7f4a2402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.789+0000 7f4a39a00640 1 -- 172.21.15.26:0/2411426516 shutdown_connections 2024-04-26T21:08:51.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.789+0000 7f4a39a00640 1 --2- 172.21.15.26:0/2411426516 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a340d1210 0x7f4a34072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.789+0000 7f4a39a00640 1 --2- 172.21.15.26:0/2411426516 >> v2:172.21.15.86:3300/0 conn(0x7f4a3406ec20 0x7f4a340d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.789+0000 7f4a39a00640 1 --2- 172.21.15.26:0/2411426516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a34146990 0x7f4a340b3f40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.789+0000 7f4a39a00640 1 -- 172.21.15.26:0/2411426516 >> 172.21.15.26:0/2411426516 conn(0x7f4a340cd840 msgr2=0x7f4a340cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:51.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.789+0000 7f4a39a00640 1 -- 172.21.15.26:0/2411426516 shutdown_connections 2024-04-26T21:08:51.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.789+0000 7f4a39a00640 1 -- 172.21.15.26:0/2411426516 wait complete. 2024-04-26T21:08:51.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.791+0000 7f4a39a00640 1 Processor -- start 2024-04-26T21:08:51.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.791+0000 7f4a39a00640 1 -- start start 2024-04-26T21:08:51.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.792+0000 7f4a39a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a3406ec20 0x7f4a340aa440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:51.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.792+0000 7f4a39a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a340d1210 0x7f4a340ab880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:51.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.792+0000 7f4a39a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a34146990 0x7f4a340ae530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:51.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.792+0000 7f4a39a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3406e420 con 0x7f4a3406ec20 2024-04-26T21:08:51.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.792+0000 7f4a39a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a34074130 con 0x7f4a340d1210 2024-04-26T21:08:51.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.792+0000 7f4a39a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4a3406e700 con 0x7f4a34146990 2024-04-26T21:08:51.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.792+0000 7f4a32a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a3406ec20 0x7f4a340aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.792+0000 7f4a32a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a3406ec20 0x7f4a340aa440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34838/0 (socket says 172.21.15.26:34838) 2024-04-26T21:08:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.792+0000 7f4a32a00640 1 -- 172.21.15.26:0/130171456 learned_addr learned my addr 172.21.15.26:0/130171456 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.792+0000 7f4a33400640 1 --2- 172.21.15.26:0/130171456 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a34146990 0x7f4a340ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.793+0000 7f4a32a00640 1 -- 172.21.15.26:0/130171456 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a34146990 msgr2=0x7f4a340ae530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.793+0000 7f4a32000640 1 --2- 172.21.15.26:0/130171456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a340d1210 0x7f4a340ab880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.793+0000 7f4a32a00640 1 --2- 172.21.15.26:0/130171456 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a34146990 0x7f4a340ae530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.793+0000 7f4a32a00640 1 -- 172.21.15.26:0/130171456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a340d1210 msgr2=0x7f4a340ab880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.793+0000 7f4a32a00640 1 --2- 172.21.15.26:0/130171456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a340d1210 0x7f4a340ab880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.793+0000 7f4a32a00640 1 -- 172.21.15.26:0/130171456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a340ab3a0 con 0x7f4a3406ec20 2024-04-26T21:08:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.793+0000 7f4a33400640 1 --2- 172.21.15.26:0/130171456 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a34146990 0x7f4a340ae530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.793+0000 7f4a32a00640 1 --2- 172.21.15.26:0/130171456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a3406ec20 0x7f4a340aa440 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4a2402a220 tx=0x7f4a240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.793+0000 7f4a2aa00640 1 -- 172.21.15.26:0/130171456 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a24041070 con 0x7f4a3406ec20 2024-04-26T21:08:51.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.793+0000 7f4a2aa00640 1 -- 172.21.15.26:0/130171456 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a2403c870 con 0x7f4a3406ec20 2024-04-26T21:08:51.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.793+0000 7f4a39a00640 1 -- 172.21.15.26:0/130171456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a340aab00 con 0x7f4a3406ec20 2024-04-26T21:08:51.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.793+0000 7f4a2aa00640 1 -- 172.21.15.26:0/130171456 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a24037040 con 0x7f4a3406ec20 2024-04-26T21:08:51.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.794+0000 7f4a39a00640 1 -- 172.21.15.26:0/130171456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a340ab0c0 con 0x7f4a3406ec20 2024-04-26T21:08:51.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.795+0000 7f4a39a00640 1 -- 172.21.15.26:0/130171456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a3406f0b0 con 0x7f4a3406ec20 2024-04-26T21:08:51.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.796+0000 7f4a2aa00640 1 -- 172.21.15.26:0/130171456 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a24044070 con 0x7f4a3406ec20 2024-04-26T21:08:51.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.797+0000 7f4a2aa00640 1 --2- 172.21.15.26:0/130171456 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4a1406c920 0x7f4a1406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:51.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.797+0000 7f4a2aa00640 1 -- 172.21.15.26:0/130171456 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4a240ac890 con 0x7f4a3406ec20 2024-04-26T21:08:51.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.797+0000 7f4a32000640 1 --2- 172.21.15.26:0/130171456 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4a1406c920 0x7f4a1406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:51.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.798+0000 7f4a32000640 1 --2- 172.21.15.26:0/130171456 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4a1406c920 0x7f4a1406ede0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4a34160b70 tx=0x7f4a20008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:51.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:51.801+0000 7f4a2aa00640 1 -- 172.21.15.26:0/130171456 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4a3406f0b0 con 0x7f4a3406ec20 2024-04-26T21:08:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.006+0000 7f4a39a00640 1 -- 172.21.15.26:0/130171456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a3406a700 con 0x7f4a3406ec20 2024-04-26T21:08:52.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.009+0000 7f4a2aa00640 1 -- 172.21.15.26:0/130171456 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16557 (secure 0 0 0) 0x7f4a2407ad40 con 0x7f4a3406ec20 2024-04-26T21:08:52.010 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:52.011 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:08:39.177854+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:08:35.481169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:08:36.609498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:08:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.012+0000 7f4a39a00640 1 -- 172.21.15.26:0/130171456 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4a1406c920 msgr2=0x7f4a1406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.012+0000 7f4a39a00640 1 --2- 172.21.15.26:0/130171456 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4a1406c920 0x7f4a1406ede0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4a34160b70 tx=0x7f4a20008040 comp rx=0 tx=0).stop 2024-04-26T21:08:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.012+0000 7f4a39a00640 1 -- 172.21.15.26:0/130171456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a3406ec20 msgr2=0x7f4a340aa440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.012+0000 7f4a39a00640 1 --2- 172.21.15.26:0/130171456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a3406ec20 0x7f4a340aa440 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4a2402a220 tx=0x7f4a240046c0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.012+0000 7f4a39a00640 1 -- 172.21.15.26:0/130171456 shutdown_connections 2024-04-26T21:08:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.012+0000 7f4a39a00640 1 --2- 172.21.15.26:0/130171456 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4a1406c920 0x7f4a1406ede0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.012+0000 7f4a39a00640 1 --2- 172.21.15.26:0/130171456 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a34146990 0x7f4a340ae530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.012+0000 7f4a39a00640 1 --2- 172.21.15.26:0/130171456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a340d1210 0x7f4a340ab880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.012+0000 7f4a39a00640 1 --2- 172.21.15.26:0/130171456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a3406ec20 0x7f4a340aa440 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.012+0000 7f4a39a00640 1 -- 172.21.15.26:0/130171456 >> 172.21.15.26:0/130171456 conn(0x7f4a340cd840 msgr2=0x7f4a340b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.012+0000 7f4a39a00640 1 -- 172.21.15.26:0/130171456 shutdown_connections 2024-04-26T21:08:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.012+0000 7f4a39a00640 1 -- 172.21.15.26:0/130171456 wait complete. 2024-04-26T21:08:52.031 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:08:31.322511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:08:35.481169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:08:36.609498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:08:52.031 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:08:52.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.245+0000 7f4608e00640 1 Processor -- start 2024-04-26T21:08:52.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.245+0000 7f4608e00640 1 -- start start 2024-04-26T21:08:52.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.246+0000 7f4608e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f460415f290 0x7f46040d5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:52.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.246+0000 7f4608e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f46040d5c40 0x7f4604162e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:52.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.246+0000 7f4608e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f46041633a0 0x7f46040cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:52.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.246+0000 7f4608e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f4604163810 con 0x7f46040d5c40 2024-04-26T21:08:52.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.246+0000 7f4608e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f46040d3870 con 0x7f46041633a0 2024-04-26T21:08:52.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.246+0000 7f4608e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f46040d3b50 con 0x7f460415f290 2024-04-26T21:08:52.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.246+0000 7f45fbe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f46040d5c40 0x7f4604162e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:52.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.246+0000 7f45fbe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f46040d5c40 0x7f4604162e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34848/0 (socket says 172.21.15.26:34848) 2024-04-26T21:08:52.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.246+0000 7f45fbe00640 1 -- 172.21.15.26:0/2769320636 learned_addr learned my addr 172.21.15.26:0/2769320636 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:52.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.247+0000 7f4603e00640 1 --2- 172.21.15.26:0/2769320636 >> v2:172.21.15.86:3300/0 conn(0x7f46041633a0 0x7f46040cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:52.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.247+0000 7f4603400640 1 --2- 172.21.15.26:0/2769320636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f460415f290 0x7f46040d5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:52.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.247+0000 7f45fbe00640 1 -- 172.21.15.26:0/2769320636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f460415f290 msgr2=0x7f46040d5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:52.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.247+0000 7f45fbe00640 1 --2- 172.21.15.26:0/2769320636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f460415f290 0x7f46040d5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.247+0000 7f45fbe00640 1 -- 172.21.15.26:0/2769320636 >> v2:172.21.15.86:3300/0 conn(0x7f46041633a0 msgr2=0x7f46040cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:52.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.247+0000 7f45fbe00640 1 --2- 172.21.15.26:0/2769320636 >> v2:172.21.15.86:3300/0 conn(0x7f46041633a0 0x7f46040cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.247+0000 7f45fbe00640 1 -- 172.21.15.26:0/2769320636 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46040d3e30 con 0x7f46040d5c40 2024-04-26T21:08:52.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.248+0000 7f45fbe00640 1 --2- 172.21.15.26:0/2769320636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040d5c40 0x7f4604162e60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f46040d4e80 tx=0x7f45ec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=defc604b9a3dabf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:52.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.248+0000 7f4602a00640 1 -- 172.21.15.26:0/2769320636 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec037070 con 0x7f46040d5c40 2024-04-26T21:08:52.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.248+0000 7f4602a00640 1 -- 172.21.15.26:0/2769320636 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45ec032440 con 0x7f46040d5c40 2024-04-26T21:08:52.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.248+0000 7f4602a00640 1 -- 172.21.15.26:0/2769320636 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec032810 con 0x7f46040d5c40 2024-04-26T21:08:52.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.249+0000 7f4608e00640 1 -- 172.21.15.26:0/2769320636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040d5c40 msgr2=0x7f4604162e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:52.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.249+0000 7f4608e00640 1 --2- 172.21.15.26:0/2769320636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040d5c40 0x7f4604162e60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f46040d4e80 tx=0x7f45ec02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.249+0000 7f4608e00640 1 -- 172.21.15.26:0/2769320636 shutdown_connections 2024-04-26T21:08:52.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.249+0000 7f4608e00640 1 --2- 172.21.15.26:0/2769320636 >> v2:172.21.15.86:3300/0 conn(0x7f46041633a0 0x7f46040cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.249+0000 7f4608e00640 1 --2- 172.21.15.26:0/2769320636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040d5c40 0x7f4604162e60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.249+0000 7f4608e00640 1 --2- 172.21.15.26:0/2769320636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f460415f290 0x7f46040d5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.249+0000 7f4608e00640 1 -- 172.21.15.26:0/2769320636 >> 172.21.15.26:0/2769320636 conn(0x7f460409e380 msgr2=0x7f46040d4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:52.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.249+0000 7f4608e00640 1 -- 172.21.15.26:0/2769320636 shutdown_connections 2024-04-26T21:08:52.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.249+0000 7f4608e00640 1 -- 172.21.15.26:0/2769320636 wait complete. 2024-04-26T21:08:52.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.251+0000 7f4608e00640 1 Processor -- start 2024-04-26T21:08:52.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.251+0000 7f4608e00640 1 -- start start 2024-04-26T21:08:52.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.252+0000 7f4608e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040d5c40 0x7f4604153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:52.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.252+0000 7f4608e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f460415f290 0x7f46040d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:52.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.252+0000 7f4608e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46041633a0 0x7f46040cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:52.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.252+0000 7f4608e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f46040d3540 con 0x7f46040d5c40 2024-04-26T21:08:52.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.252+0000 7f4608e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46040b8120 con 0x7f46041633a0 2024-04-26T21:08:52.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.252+0000 7f4608e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f460415f670 con 0x7f460415f290 2024-04-26T21:08:52.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.252+0000 7f4603400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040d5c40 0x7f4604153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:52.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.252+0000 7f4603400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040d5c40 0x7f4604153240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34858/0 (socket says 172.21.15.26:34858) 2024-04-26T21:08:52.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.252+0000 7f4603400640 1 -- 172.21.15.26:0/3967819269 learned_addr learned my addr 172.21.15.26:0/3967819269 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:52.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.252+0000 7f45fbe00640 1 --2- 172.21.15.26:0/3967819269 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f460415f290 0x7f46040d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:52.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.252+0000 7f4603e00640 1 --2- 172.21.15.26:0/3967819269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46041633a0 0x7f46040cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.253+0000 7f4603400640 1 -- 172.21.15.26:0/3967819269 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f460415f290 msgr2=0x7f46040d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.253+0000 7f4603400640 1 --2- 172.21.15.26:0/3967819269 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f460415f290 0x7f46040d04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.253+0000 7f4603400640 1 -- 172.21.15.26:0/3967819269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46041633a0 msgr2=0x7f46040cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.253+0000 7f4603400640 1 --2- 172.21.15.26:0/3967819269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46041633a0 0x7f46040cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.253+0000 7f4603400640 1 -- 172.21.15.26:0/3967819269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46040d0d20 con 0x7f46040d5c40 2024-04-26T21:08:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.253+0000 7f45fbe00640 1 --2- 172.21.15.26:0/3967819269 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f460415f290 0x7f46040d04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.253+0000 7f4603e00640 1 --2- 172.21.15.26:0/3967819269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46041633a0 0x7f46040cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.253+0000 7f4603400640 1 --2- 172.21.15.26:0/3967819269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040d5c40 0x7f4604153240 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f45f400d800 tx=0x7f45f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.253+0000 7f45fb400640 1 -- 172.21.15.26:0/3967819269 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4018020 con 0x7f46040d5c40 2024-04-26T21:08:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.253+0000 7f45fb400640 1 -- 172.21.15.26:0/3967819269 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f40096a0 con 0x7f46040d5c40 2024-04-26T21:08:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.253+0000 7f45fb400640 1 -- 172.21.15.26:0/3967819269 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4009a70 con 0x7f46040d5c40 2024-04-26T21:08:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.253+0000 7f4608e00640 1 -- 172.21.15.26:0/3967819269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46040d1000 con 0x7f46040d5c40 2024-04-26T21:08:52.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.253+0000 7f4608e00640 1 -- 172.21.15.26:0/3967819269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46040d0a10 con 0x7f46040d5c40 2024-04-26T21:08:52.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.255+0000 7f4608e00640 1 -- 172.21.15.26:0/3967819269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f460406a700 con 0x7f46040d5c40 2024-04-26T21:08:52.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.255+0000 7f45fb400640 1 -- 172.21.15.26:0/3967819269 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45f401b020 con 0x7f46040d5c40 2024-04-26T21:08:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.256+0000 7f45fb400640 1 --2- 172.21.15.26:0/3967819269 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f45e006c920 0x7f45e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.256+0000 7f45fb400640 1 -- 172.21.15.26:0/3967819269 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f45f408bda0 con 0x7f46040d5c40 2024-04-26T21:08:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.257+0000 7f45fbe00640 1 --2- 172.21.15.26:0/3967819269 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f45e006c920 0x7f45e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.257+0000 7f45fbe00640 1 --2- 172.21.15.26:0/3967819269 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f45e006c920 0x7f45e006ede0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f45ec034000 tx=0x7f45ec02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:52.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.261+0000 7f45fb400640 1 -- 172.21.15.26:0/3967819269 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f45f4022030 con 0x7f46040d5c40 2024-04-26T21:08:52.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.465+0000 7f4608e00640 1 -- 172.21.15.26:0/3967819269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f46040b76e0 con 0x7f46040d5c40 2024-04-26T21:08:52.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.466+0000 7f45fb400640 1 -- 172.21.15.26:0/3967819269 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f45f405a1c0 con 0x7f46040d5c40 2024-04-26T21:08:52.467 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-26T21:08:52.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.468+0000 7f4608e00640 1 -- 172.21.15.26:0/3967819269 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f45e006c920 msgr2=0x7f45e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:52.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.468+0000 7f4608e00640 1 --2- 172.21.15.26:0/3967819269 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f45e006c920 0x7f45e006ede0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f45ec034000 tx=0x7f45ec02e000 comp rx=0 tx=0).stop 2024-04-26T21:08:52.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.468+0000 7f4608e00640 1 -- 172.21.15.26:0/3967819269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040d5c40 msgr2=0x7f4604153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:52.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.468+0000 7f4608e00640 1 --2- 172.21.15.26:0/3967819269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040d5c40 0x7f4604153240 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f45f400d800 tx=0x7f45f400dcc0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.468+0000 7f4608e00640 1 -- 172.21.15.26:0/3967819269 shutdown_connections 2024-04-26T21:08:52.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.468+0000 7f4608e00640 1 --2- 172.21.15.26:0/3967819269 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f45e006c920 0x7f45e006ede0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.468+0000 7f4608e00640 1 --2- 172.21.15.26:0/3967819269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46041633a0 0x7f46040cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.468+0000 7f4608e00640 1 --2- 172.21.15.26:0/3967819269 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f460415f290 0x7f46040d04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.468+0000 7f4608e00640 1 --2- 172.21.15.26:0/3967819269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040d5c40 0x7f4604153240 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.468+0000 7f4608e00640 1 -- 172.21.15.26:0/3967819269 >> 172.21.15.26:0/3967819269 conn(0x7f460409e380 msgr2=0x7f4604161d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:52.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.469+0000 7f4608e00640 1 -- 172.21.15.26:0/3967819269 shutdown_connections 2024-04-26T21:08:52.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.469+0000 7f4608e00640 1 -- 172.21.15.26:0/3967819269 wait complete. 2024-04-26T21:08:52.490 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:08:52.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.699+0000 7ff8faa00640 1 Processor -- start 2024-04-26T21:08:52.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.700+0000 7ff8faa00640 1 -- start start 2024-04-26T21:08:52.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.701+0000 7ff8faa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff8f4160a80 0x7ff8f40b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:52.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.701+0000 7ff8faa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8f4072490 0x7ff8f40b6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:52.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.701+0000 7ff8faa00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f4072d90 con 0x7ff8f4160e60 2024-04-26T21:08:52.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.701+0000 7ff8faa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff8f40b5be0 con 0x7ff8f4160a80 2024-04-26T21:08:52.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.701+0000 7ff8faa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f4073070 con 0x7ff8f4072490 2024-04-26T21:08:52.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.702+0000 7ff8f8c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff8f4160e60 0x7ff8f406e210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55176/0 (socket says 172.21.15.26:55176) 2024-04-26T21:08:52.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.702+0000 7ff8f8c00640 1 -- 172.21.15.26:0/2956375321 learned_addr learned my addr 172.21.15.26:0/2956375321 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:52.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.702+0000 7ff8f9600640 1 --2- 172.21.15.26:0/2956375321 >> v2:172.21.15.86:3300/0 conn(0x7ff8f4160a80 0x7ff8f40b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:52.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.702+0000 7ff8fa000640 1 --2- 172.21.15.26:0/2956375321 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8f4072490 0x7ff8f40b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:52.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.702+0000 7ff8f3e00640 1 -- 172.21.15.26:0/2956375321 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4217531089 0 0) 0x7ff8f4072d90 con 0x7ff8f4160e60 2024-04-26T21:08:52.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.702+0000 7ff8f3e00640 1 -- 172.21.15.26:0/2956375321 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8dc003520 con 0x7ff8f4160e60 2024-04-26T21:08:52.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.702+0000 7ff8f3e00640 1 -- 172.21.15.26:0/2956375321 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1231709923 0 0) 0x7ff8dc003520 con 0x7ff8f4160e60 2024-04-26T21:08:52.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.703+0000 7ff8f3e00640 1 -- 172.21.15.26:0/2956375321 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8f4072490 msgr2=0x7ff8f40b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:52.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.703+0000 7ff8f3e00640 1 --2- 172.21.15.26:0/2956375321 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8f4072490 0x7ff8f40b6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.703+0000 7ff8f3e00640 1 -- 172.21.15.26:0/2956375321 >> v2:172.21.15.86:3300/0 conn(0x7ff8f4160a80 msgr2=0x7ff8f40b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:52.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.703+0000 7ff8f3e00640 1 --2- 172.21.15.26:0/2956375321 >> v2:172.21.15.86:3300/0 conn(0x7ff8f4160a80 0x7ff8f40b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.703+0000 7ff8f3e00640 1 -- 172.21.15.26:0/2956375321 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f4073350 con 0x7ff8f4160e60 2024-04-26T21:08:52.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.703+0000 7ff8f3e00640 1 -- 172.21.15.26:0/2956375321 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7ff8e0001670 con 0x7ff8f4160e60 2024-04-26T21:08:52.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.703+0000 7ff8f3e00640 1 -- 172.21.15.26:0/2956375321 >> v1:172.21.15.26:6789/0 conn(0x7ff8f4160e60 legacy=0x7ff8f406e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:52.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.703+0000 7ff8f3e00640 1 --2- 172.21.15.26:0/2956375321 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8f4072490 0x7ff8dc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:52.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.703+0000 7ff8f3e00640 1 --2- 172.21.15.26:0/2956375321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8dc003f80 0x7ff8dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:52.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.703+0000 7ff8f3e00640 1 --2- 172.21.15.26:0/2956375321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8dc004d40 0x7ff8dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:52.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.703+0000 7ff8f3e00640 1 -- 172.21.15.26:0/2956375321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f40b5be0 con 0x7ff8dc004d40 2024-04-26T21:08:52.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.703+0000 7ff8f3e00640 1 -- 172.21.15.26:0/2956375321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f4073070 con 0x7ff8dc003f80 2024-04-26T21:08:52.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.703+0000 7ff8f3e00640 1 -- 172.21.15.26:0/2956375321 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff8dc00b9c0 con 0x7ff8f4072490 2024-04-26T21:08:52.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.703+0000 7ff8f9600640 1 --2- 172.21.15.26:0/2956375321 >> v2:172.21.15.86:3300/0 conn(0x7ff8f4160a80 0x7ff8f40b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:08:52.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.703+0000 7ff8fa000640 1 --2- 172.21.15.26:0/2956375321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8dc004d40 0x7ff8dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:52.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.703+0000 7ff8fa000640 1 -- 172.21.15.26:0/2956375321 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8f4072490 msgr2=0x7ff8dc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:52.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.703+0000 7ff8fa000640 1 --2- 172.21.15.26:0/2956375321 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8f4072490 0x7ff8dc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.703+0000 7ff8fa000640 1 -- 172.21.15.26:0/2956375321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8dc003f80 msgr2=0x7ff8dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:52.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.703+0000 7ff8fa000640 1 --2- 172.21.15.26:0/2956375321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8dc003f80 0x7ff8dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.703+0000 7ff8fa000640 1 -- 172.21.15.26:0/2956375321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8dc003520 con 0x7ff8dc004d40 2024-04-26T21:08:52.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.704+0000 7ff8fa000640 1 --2- 172.21.15.26:0/2956375321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8dc004d40 0x7ff8dc0051c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00e170 tx=0x7ff8dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624050d6cff6ce87 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:52.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.705+0000 7ff8f3e00640 1 -- 172.21.15.26:0/2956375321 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec034020 con 0x7ff8dc004d40 2024-04-26T21:08:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.705+0000 7ff8f3e00640 1 -- 172.21.15.26:0/2956375321 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8ec030430 con 0x7ff8dc004d40 2024-04-26T21:08:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.705+0000 7ff8f3e00640 1 -- 172.21.15.26:0/2956375321 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec009c10 con 0x7ff8dc004d40 2024-04-26T21:08:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.705+0000 7ff8faa00640 1 -- 172.21.15.26:0/2956375321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8dc004d40 msgr2=0x7ff8dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.705+0000 7ff8faa00640 1 --2- 172.21.15.26:0/2956375321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8dc004d40 0x7ff8dc0051c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00e170 tx=0x7ff8dc005830 comp rx=0 tx=0).stop 2024-04-26T21:08:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.705+0000 7ff8fa000640 1 -- 172.21.15.26:0/2956375321 reap_dead start 2024-04-26T21:08:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.705+0000 7ff8faa00640 1 -- 172.21.15.26:0/2956375321 shutdown_connections 2024-04-26T21:08:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.705+0000 7ff8faa00640 1 -- 172.21.15.26:0/2956375321 >> 172.21.15.26:0/2956375321 conn(0x7ff8f40b2a10 msgr2=0x7ff8f40b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.706+0000 7ff8faa00640 1 -- 172.21.15.26:0/2956375321 shutdown_connections 2024-04-26T21:08:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.706+0000 7ff8faa00640 1 -- 172.21.15.26:0/2956375321 wait complete. 2024-04-26T21:08:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.708+0000 7ff8faa00640 1 Processor -- start 2024-04-26T21:08:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.708+0000 7ff8faa00640 1 -- start start 2024-04-26T21:08:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.709+0000 7ff8faa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8dc004d40 0x7ff8f40b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.709+0000 7ff8faa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f41f5e10 0x7ff8f415fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.709+0000 7ff8faa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8f4159650 0x7ff8f41f8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.709+0000 7ff8faa00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f4072920 con 0x7ff8f41f5e10 2024-04-26T21:08:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.709+0000 7ff8faa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41612d0 con 0x7ff8dc004d40 2024-04-26T21:08:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.709+0000 7ff8faa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f4072aa0 con 0x7ff8f4159650 2024-04-26T21:08:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.709+0000 7ff8f8c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f41f5e10 0x7ff8f415fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.709+0000 7ff8f8c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f41f5e10 0x7ff8f415fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34872/0 (socket says 172.21.15.26:34872) 2024-04-26T21:08:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.709+0000 7ff8f8c00640 1 -- 172.21.15.26:0/2953166412 learned_addr learned my addr 172.21.15.26:0/2953166412 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.710+0000 7ff8fa000640 1 --2- 172.21.15.26:0/2953166412 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8f4159650 0x7ff8f41f8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.710+0000 7ff8f9600640 1 --2- 172.21.15.26:0/2953166412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8dc004d40 0x7ff8f40b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:52.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.710+0000 7ff8f8c00640 1 -- 172.21.15.26:0/2953166412 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8f4159650 msgr2=0x7ff8f41f8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:52.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.710+0000 7ff8f8c00640 1 --2- 172.21.15.26:0/2953166412 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8f4159650 0x7ff8f41f8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.710+0000 7ff8f8c00640 1 -- 172.21.15.26:0/2953166412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8dc004d40 msgr2=0x7ff8f40b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:52.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.710+0000 7ff8f8c00640 1 --2- 172.21.15.26:0/2953166412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8dc004d40 0x7ff8f40b2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.710+0000 7ff8f8c00640 1 -- 172.21.15.26:0/2953166412 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f41f8a00 con 0x7ff8f41f5e10 2024-04-26T21:08:52.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.710+0000 7ff8f8c00640 1 --2- 172.21.15.26:0/2953166412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f41f5e10 0x7ff8f415fac0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00011e0 tx=0x7ff8e000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:52.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.711+0000 7ff8f1600640 1 -- 172.21.15.26:0/2953166412 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0018070 con 0x7ff8f41f5e10 2024-04-26T21:08:52.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.711+0000 7ff8f1600640 1 -- 172.21.15.26:0/2953166412 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e000ce50 con 0x7ff8f41f5e10 2024-04-26T21:08:52.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.711+0000 7ff8f1600640 1 -- 172.21.15.26:0/2953166412 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e00047e0 con 0x7ff8f41f5e10 2024-04-26T21:08:52.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.711+0000 7ff8faa00640 1 -- 172.21.15.26:0/2953166412 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f41f6290 con 0x7ff8f41f5e10 2024-04-26T21:08:52.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.711+0000 7ff8faa00640 1 -- 172.21.15.26:0/2953166412 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f40d0540 con 0x7ff8f41f5e10 2024-04-26T21:08:52.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.712+0000 7ff8faa00640 1 -- 172.21.15.26:0/2953166412 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f41f8bd0 con 0x7ff8f41f5e10 2024-04-26T21:08:52.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.713+0000 7ff8f1600640 1 -- 172.21.15.26:0/2953166412 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8e0004980 con 0x7ff8f41f5e10 2024-04-26T21:08:52.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.714+0000 7ff8f1600640 1 --2- 172.21.15.26:0/2953166412 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff8d406c8f0 0x7ff8d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:52.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.714+0000 7ff8f1600640 1 -- 172.21.15.26:0/2953166412 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff8e008b090 con 0x7ff8f41f5e10 2024-04-26T21:08:52.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.714+0000 7ff8f9600640 1 --2- 172.21.15.26:0/2953166412 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff8d406c8f0 0x7ff8d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:52.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.715+0000 7ff8f9600640 1 --2- 172.21.15.26:0/2953166412 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff8d406c8f0 0x7ff8d406edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8004790 tx=0x7ff8e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:52.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.718+0000 7ff8f1600640 1 -- 172.21.15.26:0/2953166412 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff8e0059420 con 0x7ff8f41f5e10 2024-04-26T21:08:52.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.911+0000 7ff8faa00640 1 -- 172.21.15.26:0/2953166412 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff8f406a700 con 0x7ff8f41f5e10 2024-04-26T21:08:52.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.912+0000 7ff8f1600640 1 -- 172.21.15.26:0/2953166412 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7ff8e0058d20 con 0x7ff8f41f5e10 2024-04-26T21:08:52.914 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-26T21:08:52.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.914+0000 7ff8faa00640 1 -- 172.21.15.26:0/2953166412 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff8d406c8f0 msgr2=0x7ff8d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:52.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.914+0000 7ff8faa00640 1 --2- 172.21.15.26:0/2953166412 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff8d406c8f0 0x7ff8d406edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8004790 tx=0x7ff8e8008040 comp rx=0 tx=0).stop 2024-04-26T21:08:52.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.914+0000 7ff8faa00640 1 -- 172.21.15.26:0/2953166412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f41f5e10 msgr2=0x7ff8f415fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:52.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.914+0000 7ff8faa00640 1 --2- 172.21.15.26:0/2953166412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f41f5e10 0x7ff8f415fac0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00011e0 tx=0x7ff8e000cc00 comp rx=0 tx=0).stop 2024-04-26T21:08:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.915+0000 7ff8faa00640 1 -- 172.21.15.26:0/2953166412 shutdown_connections 2024-04-26T21:08:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.915+0000 7ff8faa00640 1 --2- 172.21.15.26:0/2953166412 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff8d406c8f0 0x7ff8d406edb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.915+0000 7ff8faa00640 1 --2- 172.21.15.26:0/2953166412 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8f4159650 0x7ff8f41f8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.915+0000 7ff8faa00640 1 --2- 172.21.15.26:0/2953166412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f41f5e10 0x7ff8f415fac0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.915+0000 7ff8faa00640 1 --2- 172.21.15.26:0/2953166412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8dc004d40 0x7ff8f40b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.915+0000 7ff8faa00640 1 -- 172.21.15.26:0/2953166412 >> 172.21.15.26:0/2953166412 conn(0x7ff8f40b2a10 msgr2=0x7ff8f4073930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.915+0000 7ff8faa00640 1 -- 172.21.15.26:0/2953166412 shutdown_connections 2024-04-26T21:08:52.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:52.915+0000 7ff8faa00640 1 -- 172.21.15.26:0/2953166412 wait complete. 2024-04-26T21:08:52.934 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:08:53.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.140+0000 7fc2e3200640 1 Processor -- start 2024-04-26T21:08:53.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.140+0000 7fc2e3200640 1 -- start start 2024-04-26T21:08:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.142+0000 7fc2e3200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc2e4146a30 0x7fc2e40b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.142+0000 7fc2e3200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2e406f9f0 0x7fc2e40cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.142+0000 7fc2e3200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2e40b42c0 con 0x7fc2e415d590 2024-04-26T21:08:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.142+0000 7fc2e3200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fc2e40cebd0 con 0x7fc2e4146a30 2024-04-26T21:08:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.142+0000 7fc2e3200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc2e40ceeb0 con 0x7fc2e406f9f0 2024-04-26T21:08:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.142+0000 7fc2e1e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc2e4146a30 0x7fc2e40b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.142+0000 7fc2e1e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc2e4146a30 0x7fc2e40b3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34886/0 (socket says 172.21.15.26:34886) 2024-04-26T21:08:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.142+0000 7fc2e1e00640 1 -- 172.21.15.26:0/2947275596 learned_addr learned my addr 172.21.15.26:0/2947275596 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.142+0000 7fc2e2800640 1 --2- 172.21.15.26:0/2947275596 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2e406f9f0 0x7fc2e40cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.143+0000 7fc2e1e00640 1 -- 172.21.15.26:0/2947275596 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2e406f9f0 msgr2=0x7fc2e40cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.143+0000 7fc2e1e00640 1 --2- 172.21.15.26:0/2947275596 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2e406f9f0 0x7fc2e40cff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.143+0000 7fc2e1e00640 1 -- 172.21.15.26:0/2947275596 >> v1:172.21.15.86:6789/0 conn(0x7fc2e415d590 legacy=0x7fc2e406f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:53.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.143+0000 7fc2e1e00640 1 -- 172.21.15.26:0/2947275596 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e40cf190 con 0x7fc2e4146a30 2024-04-26T21:08:53.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.144+0000 7fc2e1e00640 1 --2- 172.21.15.26:0/2947275596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2e4146a30 0x7fc2e40b3d80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009860 tx=0x7fc2cc02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f36b22c82feb538 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:53.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.145+0000 7fc2e0a00640 1 -- 172.21.15.26:0/2947275596 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc037070 con 0x7fc2e4146a30 2024-04-26T21:08:53.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.145+0000 7fc2e0a00640 1 -- 172.21.15.26:0/2947275596 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2cc032440 con 0x7fc2e4146a30 2024-04-26T21:08:53.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.145+0000 7fc2e0a00640 1 -- 172.21.15.26:0/2947275596 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc032810 con 0x7fc2e4146a30 2024-04-26T21:08:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.145+0000 7fc2e3200640 1 -- 172.21.15.26:0/2947275596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2e4146a30 msgr2=0x7fc2e40b3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.145+0000 7fc2e3200640 1 --2- 172.21.15.26:0/2947275596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2e4146a30 0x7fc2e40b3d80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009860 tx=0x7fc2cc02a250 comp rx=0 tx=0).stop 2024-04-26T21:08:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.146+0000 7fc2e3200640 1 -- 172.21.15.26:0/2947275596 shutdown_connections 2024-04-26T21:08:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.146+0000 7fc2e3200640 1 --2- 172.21.15.26:0/2947275596 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2e406f9f0 0x7fc2e40cff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.146+0000 7fc2e3200640 1 --2- 172.21.15.26:0/2947275596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2e4146a30 0x7fc2e40b3d80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.146+0000 7fc2e3200640 1 -- 172.21.15.26:0/2947275596 >> 172.21.15.26:0/2947275596 conn(0x7fc2e40cd7d0 msgr2=0x7fc2e40cdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.146+0000 7fc2e3200640 1 -- 172.21.15.26:0/2947275596 shutdown_connections 2024-04-26T21:08:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.146+0000 7fc2e3200640 1 -- 172.21.15.26:0/2947275596 wait complete. 2024-04-26T21:08:53.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.147+0000 7fc2e3200640 1 Processor -- start 2024-04-26T21:08:53.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.148+0000 7fc2e3200640 1 -- start start 2024-04-26T21:08:53.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.149+0000 7fc2e3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2e406f9f0 0x7fc2e4145520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:53.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.149+0000 7fc2e3200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2e4146a30 0x7fc2e40afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:53.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.149+0000 7fc2e3200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2e415d590 0x7fc2e41417c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:53.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.149+0000 7fc2e3200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e40cfa30 con 0x7fc2e415d590 2024-04-26T21:08:53.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.149+0000 7fc2e3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e40cf750 con 0x7fc2e406f9f0 2024-04-26T21:08:53.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.149+0000 7fc2e3200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc2e40b6d20 con 0x7fc2e4146a30 2024-04-26T21:08:53.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.149+0000 7fc2e2800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2e415d590 0x7fc2e41417c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:53.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.149+0000 7fc2e2800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2e415d590 0x7fc2e41417c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34892/0 (socket says 172.21.15.26:34892) 2024-04-26T21:08:53.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.149+0000 7fc2e2800640 1 -- 172.21.15.26:0/3178365 learned_addr learned my addr 172.21.15.26:0/3178365 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:53.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.149+0000 7fc2e1e00640 1 --2- 172.21.15.26:0/3178365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2e406f9f0 0x7fc2e4145520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:53.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.149+0000 7fc2e1400640 1 --2- 172.21.15.26:0/3178365 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2e4146a30 0x7fc2e40afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:53.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.149+0000 7fc2e2800640 1 -- 172.21.15.26:0/3178365 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2e4146a30 msgr2=0x7fc2e40afce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:53.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.149+0000 7fc2e2800640 1 --2- 172.21.15.26:0/3178365 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2e4146a30 0x7fc2e40afce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.149+0000 7fc2e2800640 1 -- 172.21.15.26:0/3178365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2e406f9f0 msgr2=0x7fc2e4145520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:53.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.149+0000 7fc2e2800640 1 --2- 172.21.15.26:0/3178365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2e406f9f0 0x7fc2e4145520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.149+0000 7fc2e2800640 1 -- 172.21.15.26:0/3178365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e41395c0 con 0x7fc2e415d590 2024-04-26T21:08:53.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.149+0000 7fc2e1e00640 1 --2- 172.21.15.26:0/3178365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2e406f9f0 0x7fc2e4145520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:08:53.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.150+0000 7fc2e2800640 1 --2- 172.21.15.26:0/3178365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2e415d590 0x7fc2e41417c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00efc0 tx=0x7fc2dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.150+0000 7fc2da000640 1 -- 172.21.15.26:0/3178365 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc018020 con 0x7fc2e415d590 2024-04-26T21:08:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.150+0000 7fc2da000640 1 -- 172.21.15.26:0/3178365 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2dc014480 con 0x7fc2e415d590 2024-04-26T21:08:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.150+0000 7fc2da000640 1 -- 172.21.15.26:0/3178365 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc014870 con 0x7fc2e415d590 2024-04-26T21:08:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.150+0000 7fc2e3200640 1 -- 172.21.15.26:0/3178365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e41398a0 con 0x7fc2e415d590 2024-04-26T21:08:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.150+0000 7fc2e3200640 1 -- 172.21.15.26:0/3178365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e4139e60 con 0x7fc2e415d590 2024-04-26T21:08:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.151+0000 7fc2e3200640 1 -- 172.21.15.26:0/3178365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2e40506c0 con 0x7fc2e415d590 2024-04-26T21:08:53.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.152+0000 7fc2da000640 1 -- 172.21.15.26:0/3178365 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc2dc005d10 con 0x7fc2e415d590 2024-04-26T21:08:53.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.153+0000 7fc2da000640 1 --2- 172.21.15.26:0/3178365 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc2c006c8f0 0x7fc2c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:53.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.153+0000 7fc2da000640 1 -- 172.21.15.26:0/3178365 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc2dc08caa0 con 0x7fc2e415d590 2024-04-26T21:08:53.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.153+0000 7fc2e1e00640 1 --2- 172.21.15.26:0/3178365 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc2c006c8f0 0x7fc2c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:53.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.154+0000 7fc2e1e00640 1 --2- 172.21.15.26:0/3178365 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc2c006c8f0 0x7fc2c006edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc034000 tx=0x7fc2cc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:53.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.158+0000 7fc2da000640 1 -- 172.21.15.26:0/3178365 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc2dc022030 con 0x7fc2e415d590 2024-04-26T21:08:53.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.359+0000 7fc2e3200640 1 -- 172.21.15.26:0/3178365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc2e406e5b0 con 0x7fc2e415d590 2024-04-26T21:08:53.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.360+0000 7fc2da000640 1 -- 172.21.15.26:0/3178365 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fc2dc010040 con 0x7fc2e415d590 2024-04-26T21:08:53.361 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-26T21:08:53.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.362+0000 7fc2e3200640 1 -- 172.21.15.26:0/3178365 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc2c006c8f0 msgr2=0x7fc2c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:53.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.362+0000 7fc2e3200640 1 --2- 172.21.15.26:0/3178365 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc2c006c8f0 0x7fc2c006edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc034000 tx=0x7fc2cc02e000 comp rx=0 tx=0).stop 2024-04-26T21:08:53.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.362+0000 7fc2e3200640 1 -- 172.21.15.26:0/3178365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2e415d590 msgr2=0x7fc2e41417c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:53.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.362+0000 7fc2e3200640 1 --2- 172.21.15.26:0/3178365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2e415d590 0x7fc2e41417c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00efc0 tx=0x7fc2dc00c490 comp rx=0 tx=0).stop 2024-04-26T21:08:53.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.362+0000 7fc2e3200640 1 -- 172.21.15.26:0/3178365 shutdown_connections 2024-04-26T21:08:53.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.362+0000 7fc2e3200640 1 --2- 172.21.15.26:0/3178365 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc2c006c8f0 0x7fc2c006edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.362+0000 7fc2e3200640 1 --2- 172.21.15.26:0/3178365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2e415d590 0x7fc2e41417c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.362+0000 7fc2e3200640 1 --2- 172.21.15.26:0/3178365 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2e4146a30 0x7fc2e40afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.362+0000 7fc2e3200640 1 --2- 172.21.15.26:0/3178365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2e406f9f0 0x7fc2e4145520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.362+0000 7fc2e3200640 1 -- 172.21.15.26:0/3178365 >> 172.21.15.26:0/3178365 conn(0x7fc2e40cd7d0 msgr2=0x7fc2e40b1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:53.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.362+0000 7fc2e3200640 1 -- 172.21.15.26:0/3178365 shutdown_connections 2024-04-26T21:08:53.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.362+0000 7fc2e3200640 1 -- 172.21.15.26:0/3178365 wait complete. 2024-04-26T21:08:53.382 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:08:53.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.581+0000 7f4f40a00640 1 Processor -- start 2024-04-26T21:08:53.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.582+0000 7f4f40a00640 1 -- start start 2024-04-26T21:08:53.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.583+0000 7f4f40a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4f3c162ba0 0x7f4f3c0d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:53.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.583+0000 7f4f40a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4f3c0d2060 0x7f4f3c0d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:53.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.583+0000 7f4f40a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f3c0b61c0 con 0x7f4f3c0d1c80 2024-04-26T21:08:53.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.583+0000 7f4f40a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4f3c0b5f70 con 0x7f4f3c162ba0 2024-04-26T21:08:53.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.583+0000 7f4f40a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4f3c0d3830 con 0x7f4f3c0d2060 2024-04-26T21:08:53.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.584+0000 7f4f3be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4f3c0d2060 0x7f4f3c0d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:53.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.584+0000 7f4f3be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4f3c0d2060 0x7f4f3c0d3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.26:34362/0 (socket says 172.21.15.26:34362) 2024-04-26T21:08:53.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.584+0000 7f4f3be00640 1 -- 172.21.15.26:0/3420841515 learned_addr learned my addr 172.21.15.26:0/3420841515 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:53.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.584+0000 7f4f3aa00640 1 --2- 172.21.15.26:0/3420841515 >> v2:172.21.15.86:3300/0 conn(0x7f4f3c162ba0 0x7f4f3c0d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:53.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.585+0000 7f4f3be00640 1 -- 172.21.15.26:0/3420841515 >> v2:172.21.15.86:3300/0 conn(0x7f4f3c162ba0 msgr2=0x7f4f3c0d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:53.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.585+0000 7f4f3be00640 1 --2- 172.21.15.26:0/3420841515 >> v2:172.21.15.86:3300/0 conn(0x7f4f3c162ba0 0x7f4f3c0d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.585+0000 7f4f3be00640 1 -- 172.21.15.26:0/3420841515 >> v1:172.21.15.86:6789/0 conn(0x7f4f3c0d1c80 legacy=0x7f4f3c162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:53.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.585+0000 7f4f3be00640 1 -- 172.21.15.26:0/3420841515 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3c0b64a0 con 0x7f4f3c0d2060 2024-04-26T21:08:53.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.586+0000 7f4f3be00640 1 --2- 172.21.15.26:0/3420841515 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4f3c0d2060 0x7f4f3c0d3cc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4f34009870 tx=0x7f4f3402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eff28eae7b36c8f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:53.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.587+0000 7f4f3a000640 1 -- 172.21.15.26:0/3420841515 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34037070 con 0x7f4f3c0d2060 2024-04-26T21:08:53.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.587+0000 7f4f3a000640 1 -- 172.21.15.26:0/3420841515 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f34032440 con 0x7f4f3c0d2060 2024-04-26T21:08:53.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.587+0000 7f4f3a000640 1 -- 172.21.15.26:0/3420841515 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34032810 con 0x7f4f3c0d2060 2024-04-26T21:08:53.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.587+0000 7f4f40a00640 1 -- 172.21.15.26:0/3420841515 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4f3c0d2060 msgr2=0x7f4f3c0d3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:53.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.587+0000 7f4f40a00640 1 --2- 172.21.15.26:0/3420841515 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4f3c0d2060 0x7f4f3c0d3cc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4f34009870 tx=0x7f4f3402a1c0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.587+0000 7f4f40a00640 1 -- 172.21.15.26:0/3420841515 shutdown_connections 2024-04-26T21:08:53.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.587+0000 7f4f40a00640 1 --2- 172.21.15.26:0/3420841515 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4f3c0d2060 0x7f4f3c0d3cc0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.587+0000 7f4f40a00640 1 --2- 172.21.15.26:0/3420841515 >> v2:172.21.15.86:3300/0 conn(0x7f4f3c162ba0 0x7f4f3c0d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.587+0000 7f4f40a00640 1 -- 172.21.15.26:0/3420841515 >> 172.21.15.26:0/3420841515 conn(0x7f4f3c0d5020 msgr2=0x7f4f3c0d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:53.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.587+0000 7f4f40a00640 1 -- 172.21.15.26:0/3420841515 shutdown_connections 2024-04-26T21:08:53.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.587+0000 7f4f40a00640 1 -- 172.21.15.26:0/3420841515 wait complete. 2024-04-26T21:08:53.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.589+0000 7f4f40a00640 1 Processor -- start 2024-04-26T21:08:53.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.589+0000 7f4f40a00640 1 -- start start 2024-04-26T21:08:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.590+0000 7f4f40a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3c0d1c80 0x7f4f3c150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.590+0000 7f4f40a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4f3c0d2060 0x7f4f3c06dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.590+0000 7f4f40a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f3c162ba0 0x7f4f3c06e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.590+0000 7f4f40a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c0d5cc0 con 0x7f4f3c0d1c80 2024-04-26T21:08:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.590+0000 7f4f40a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c0d5b40 con 0x7f4f3c162ba0 2024-04-26T21:08:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.590+0000 7f4f40a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4f3c0c8b70 con 0x7f4f3c0d2060 2024-04-26T21:08:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.591+0000 7f4f3b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3c0d1c80 0x7f4f3c150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.591+0000 7f4f3b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3c0d1c80 0x7f4f3c150600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34908/0 (socket says 172.21.15.26:34908) 2024-04-26T21:08:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.591+0000 7f4f3b400640 1 -- 172.21.15.26:0/1735300041 learned_addr learned my addr 172.21.15.26:0/1735300041 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.591+0000 7f4f3aa00640 1 --2- 172.21.15.26:0/1735300041 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4f3c0d2060 0x7f4f3c06dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.591+0000 7f4f3be00640 1 --2- 172.21.15.26:0/1735300041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f3c162ba0 0x7f4f3c06e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.591+0000 7f4f3b400640 1 -- 172.21.15.26:0/1735300041 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4f3c0d2060 msgr2=0x7f4f3c06dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.591+0000 7f4f3b400640 1 --2- 172.21.15.26:0/1735300041 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4f3c0d2060 0x7f4f3c06dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.591+0000 7f4f3b400640 1 -- 172.21.15.26:0/1735300041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f3c162ba0 msgr2=0x7f4f3c06e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.591+0000 7f4f3b400640 1 --2- 172.21.15.26:0/1735300041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f3c162ba0 0x7f4f3c06e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.591+0000 7f4f3b400640 1 -- 172.21.15.26:0/1735300041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3c1f54f0 con 0x7f4f3c0d1c80 2024-04-26T21:08:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.592+0000 7f4f3b400640 1 --2- 172.21.15.26:0/1735300041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3c0d1c80 0x7f4f3c150600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00c430 tx=0x7f4f2c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.592+0000 7f4f33400640 1 -- 172.21.15.26:0/1735300041 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c0052d0 con 0x7f4f3c0d1c80 2024-04-26T21:08:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.592+0000 7f4f33400640 1 -- 172.21.15.26:0/1735300041 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f2c005490 con 0x7f4f3c0d1c80 2024-04-26T21:08:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.592+0000 7f4f40a00640 1 -- 172.21.15.26:0/1735300041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f3c1f57b0 con 0x7f4f3c0d1c80 2024-04-26T21:08:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.592+0000 7f4f33400640 1 -- 172.21.15.26:0/1735300041 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c007260 con 0x7f4f3c0d1c80 2024-04-26T21:08:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.592+0000 7f4f40a00640 1 -- 172.21.15.26:0/1735300041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f3c1f5d70 con 0x7f4f3c0d1c80 2024-04-26T21:08:53.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.593+0000 7f4f40a00640 1 -- 172.21.15.26:0/1735300041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f3c06a700 con 0x7f4f3c0d1c80 2024-04-26T21:08:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.595+0000 7f4f33400640 1 -- 172.21.15.26:0/1735300041 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f2c007400 con 0x7f4f3c0d1c80 2024-04-26T21:08:53.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.596+0000 7f4f33400640 1 --2- 172.21.15.26:0/1735300041 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4f1806c8f0 0x7f4f1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:53.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.596+0000 7f4f33400640 1 -- 172.21.15.26:0/1735300041 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4f2c08b0a0 con 0x7f4f3c0d1c80 2024-04-26T21:08:53.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.596+0000 7f4f3aa00640 1 --2- 172.21.15.26:0/1735300041 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4f1806c8f0 0x7f4f1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:53.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.597+0000 7f4f3aa00640 1 --2- 172.21.15.26:0/1735300041 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4f1806c8f0 0x7f4f1806edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0b9300 tx=0x7f4f2400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:53.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.601+0000 7f4f33400640 1 -- 172.21.15.26:0/1735300041 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4f2c058de0 con 0x7f4f3c0d1c80 2024-04-26T21:08:53.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.844+0000 7f4f40a00640 1 -- 172.21.15.26:0/1735300041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f3c0b5ae0 con 0x7f4f3c0d1c80 2024-04-26T21:08:53.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.845+0000 7f4f33400640 1 -- 172.21.15.26:0/1735300041 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7f4f2c05d0e0 con 0x7f4f3c0d1c80 2024-04-26T21:08:53.847 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:53.847 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/2695371801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2695371801},{"type":"v1","addr":"172.21.15.26:6836","nonce":2695371801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3349732696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3349732696},{"type":"v1","addr":"172.21.15.26:6837","nonce":3349732696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/2288633870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2288633870},{"type":"v1","addr":"172.21.15.86:6833","nonce":2288633870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:08:37.622844+0000","modified":"2024-04-26T21:08:40.715140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/2125360615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2125360615},{"type":"v1","addr":"172.21.15.86:6835","nonce":2125360615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4261,"qdb_cluster":[4261]},"id":1}]} 2024-04-26T21:08:53.847 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2024-04-26T21:08:53.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.847+0000 7f4f40a00640 1 -- 172.21.15.26:0/1735300041 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4f1806c8f0 msgr2=0x7f4f1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:53.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.848+0000 7f4f40a00640 1 --2- 172.21.15.26:0/1735300041 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4f1806c8f0 0x7f4f1806edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0b9300 tx=0x7f4f2400b040 comp rx=0 tx=0).stop 2024-04-26T21:08:53.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.848+0000 7f4f40a00640 1 -- 172.21.15.26:0/1735300041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3c0d1c80 msgr2=0x7f4f3c150600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:53.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.848+0000 7f4f40a00640 1 --2- 172.21.15.26:0/1735300041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3c0d1c80 0x7f4f3c150600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00c430 tx=0x7f4f2c00c8f0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.848+0000 7f4f40a00640 1 -- 172.21.15.26:0/1735300041 shutdown_connections 2024-04-26T21:08:53.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.848+0000 7f4f40a00640 1 --2- 172.21.15.26:0/1735300041 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4f1806c8f0 0x7f4f1806edb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.848+0000 7f4f40a00640 1 --2- 172.21.15.26:0/1735300041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f3c162ba0 0x7f4f3c06e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.848+0000 7f4f40a00640 1 --2- 172.21.15.26:0/1735300041 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4f3c0d2060 0x7f4f3c06dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.848+0000 7f4f40a00640 1 --2- 172.21.15.26:0/1735300041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3c0d1c80 0x7f4f3c150600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:53.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.848+0000 7f4f40a00640 1 -- 172.21.15.26:0/1735300041 >> 172.21.15.26:0/1735300041 conn(0x7f4f3c0d5020 msgr2=0x7f4f3c0d3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:53.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.848+0000 7f4f40a00640 1 -- 172.21.15.26:0/1735300041 shutdown_connections 2024-04-26T21:08:53.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:53.848+0000 7f4f40a00640 1 -- 172.21.15.26:0/1735300041 wait complete. 2024-04-26T21:08:53.868 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:08:54.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.075+0000 7fd18d800640 1 Processor -- start 2024-04-26T21:08:54.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.075+0000 7fd18d800640 1 -- start start 2024-04-26T21:08:54.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.077+0000 7fd18d800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd188159770 0x7fd1880b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:54.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.077+0000 7fd18d800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd188159b50 0x7fd18815f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:54.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.077+0000 7fd18d800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd188160440 con 0x7fd18815fdd0 2024-04-26T21:08:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.077+0000 7fd18d800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fd188160220 con 0x7fd188159770 2024-04-26T21:08:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.077+0000 7fd18d800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd188160720 con 0x7fd188159b50 2024-04-26T21:08:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.077+0000 7fd187e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd188159770 0x7fd1880b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.077+0000 7fd187e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd188159770 0x7fd1880b4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34914/0 (socket says 172.21.15.26:34914) 2024-04-26T21:08:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.077+0000 7fd187e00640 1 -- 172.21.15.26:0/3115823756 learned_addr learned my addr 172.21.15.26:0/3115823756 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.078+0000 7fd17fe00640 1 --2- 172.21.15.26:0/3115823756 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd188159b50 0x7fd18815f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.078+0000 7fd187e00640 1 -- 172.21.15.26:0/3115823756 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd188159b50 msgr2=0x7fd18815f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.078+0000 7fd187e00640 1 --2- 172.21.15.26:0/3115823756 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd188159b50 0x7fd18815f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.078+0000 7fd187e00640 1 -- 172.21.15.26:0/3115823756 >> v1:172.21.15.26:6789/0 conn(0x7fd18815fdd0 legacy=0x7fd1880d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.078+0000 7fd187e00640 1 -- 172.21.15.26:0/3115823756 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd188072560 con 0x7fd188159770 2024-04-26T21:08:54.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.079+0000 7fd187e00640 1 --2- 172.21.15.26:0/3115823756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd188159770 0x7fd1880b4160 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd178009870 tx=0x7fd17802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c37fe8ccf721a5b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:54.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.079+0000 7fd187400640 1 -- 172.21.15.26:0/3115823756 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178037070 con 0x7fd188159770 2024-04-26T21:08:54.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.079+0000 7fd187400640 1 -- 172.21.15.26:0/3115823756 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd178032440 con 0x7fd188159770 2024-04-26T21:08:54.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.079+0000 7fd187400640 1 -- 172.21.15.26:0/3115823756 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178032810 con 0x7fd188159770 2024-04-26T21:08:54.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.080+0000 7fd18d800640 1 -- 172.21.15.26:0/3115823756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd188159770 msgr2=0x7fd1880b4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:54.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.080+0000 7fd18d800640 1 --2- 172.21.15.26:0/3115823756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd188159770 0x7fd1880b4160 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd178009870 tx=0x7fd17802a1c0 comp rx=0 tx=0).stop 2024-04-26T21:08:54.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.080+0000 7fd18d800640 1 -- 172.21.15.26:0/3115823756 shutdown_connections 2024-04-26T21:08:54.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.080+0000 7fd18d800640 1 --2- 172.21.15.26:0/3115823756 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd188159b50 0x7fd18815f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:54.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.080+0000 7fd18d800640 1 --2- 172.21.15.26:0/3115823756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd188159770 0x7fd1880b4160 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:54.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.080+0000 7fd18d800640 1 -- 172.21.15.26:0/3115823756 >> 172.21.15.26:0/3115823756 conn(0x7fd18806c690 msgr2=0x7fd18806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:54.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.080+0000 7fd18d800640 1 -- 172.21.15.26:0/3115823756 shutdown_connections 2024-04-26T21:08:54.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.080+0000 7fd18d800640 1 -- 172.21.15.26:0/3115823756 wait complete. 2024-04-26T21:08:54.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.082+0000 7fd18d800640 1 Processor -- start 2024-04-26T21:08:54.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.082+0000 7fd18d800640 1 -- start start 2024-04-26T21:08:54.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.083+0000 7fd18d800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd188159770 0x7fd18815ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:54.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.083+0000 7fd18d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd188159b50 0x7fd1881559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:54.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.083+0000 7fd18d800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd18815fdd0 0x7fd18815a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:54.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.083+0000 7fd18d800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd18806ee10 con 0x7fd188159770 2024-04-26T21:08:54.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.083+0000 7fd18d800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1880b46a0 con 0x7fd188159b50 2024-04-26T21:08:54.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.083+0000 7fd18d800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd1880d1840 con 0x7fd18815fdd0 2024-04-26T21:08:54.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.083+0000 7fd187e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd188159770 0x7fd18815ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:54.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.083+0000 7fd187e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd188159770 0x7fd18815ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34920/0 (socket says 172.21.15.26:34920) 2024-04-26T21:08:54.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.083+0000 7fd187e00640 1 -- 172.21.15.26:0/2073682030 learned_addr learned my addr 172.21.15.26:0/2073682030 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:54.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.083+0000 7fd187e00640 1 -- 172.21.15.26:0/2073682030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd18815fdd0 msgr2=0x7fd18815a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:54.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.083+0000 7fd187e00640 1 --2- 172.21.15.26:0/2073682030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd18815fdd0 0x7fd18815a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:54.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.083+0000 7fd187e00640 1 -- 172.21.15.26:0/2073682030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd188159b50 msgr2=0x7fd1881559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:54.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.083+0000 7fd187e00640 1 --2- 172.21.15.26:0/2073682030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd188159b50 0x7fd1881559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:54.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.083+0000 7fd187e00640 1 -- 172.21.15.26:0/2073682030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1881563f0 con 0x7fd188159770 2024-04-26T21:08:54.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.084+0000 7fd187e00640 1 --2- 172.21.15.26:0/2073682030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd188159770 0x7fd18815ea60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd178000c00 tx=0x7fd1780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:54.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.084+0000 7fd184c00640 1 -- 172.21.15.26:0/2073682030 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178041020 con 0x7fd188159770 2024-04-26T21:08:54.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.084+0000 7fd184c00640 1 -- 172.21.15.26:0/2073682030 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1780306c0 con 0x7fd188159770 2024-04-26T21:08:54.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.084+0000 7fd184c00640 1 -- 172.21.15.26:0/2073682030 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178037040 con 0x7fd188159770 2024-04-26T21:08:54.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.084+0000 7fd18d800640 1 -- 172.21.15.26:0/2073682030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd18815df80 con 0x7fd188159770 2024-04-26T21:08:54.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.084+0000 7fd18d800640 1 -- 172.21.15.26:0/2073682030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd18815dba0 con 0x7fd188159770 2024-04-26T21:08:54.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.085+0000 7fd18d800640 1 -- 172.21.15.26:0/2073682030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd18806a700 con 0x7fd188159770 2024-04-26T21:08:54.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.086+0000 7fd184c00640 1 -- 172.21.15.26:0/2073682030 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd178044020 con 0x7fd188159770 2024-04-26T21:08:54.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.087+0000 7fd184c00640 1 --2- 172.21.15.26:0/2073682030 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd16406c8f0 0x7fd16406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:54.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.087+0000 7fd184c00640 1 -- 172.21.15.26:0/2073682030 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd1780ac390 con 0x7fd188159770 2024-04-26T21:08:54.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.088+0000 7fd17fe00640 1 --2- 172.21.15.26:0/2073682030 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd16406c8f0 0x7fd16406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:54.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.088+0000 7fd17fe00640 1 --2- 172.21.15.26:0/2073682030 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd16406c8f0 0x7fd16406edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd188153120 tx=0x7fd170008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:54.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.092+0000 7fd184c00640 1 -- 172.21.15.26:0/2073682030 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd17804b030 con 0x7fd188159770 2024-04-26T21:08:54.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.304+0000 7fd18d800640 1 -- 172.21.15.26:0/2073682030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd188158be0 con 0x7fd188159770 2024-04-26T21:08:54.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.306+0000 7fd184c00640 1 -- 172.21.15.26:0/2073682030 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16557 (secure 0 0 0) 0x7fd17807a7b0 con 0x7fd188159770 2024-04-26T21:08:54.308 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:54.309 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:08:39.177854+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:08:35.481169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:08:36.609498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:08:54.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.309+0000 7fd18d800640 1 -- 172.21.15.26:0/2073682030 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd16406c8f0 msgr2=0x7fd16406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:54.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.309+0000 7fd18d800640 1 --2- 172.21.15.26:0/2073682030 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd16406c8f0 0x7fd16406edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd188153120 tx=0x7fd170008040 comp rx=0 tx=0).stop 2024-04-26T21:08:54.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.309+0000 7fd18d800640 1 -- 172.21.15.26:0/2073682030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd188159770 msgr2=0x7fd18815ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:54.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.309+0000 7fd18d800640 1 --2- 172.21.15.26:0/2073682030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd188159770 0x7fd18815ea60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd178000c00 tx=0x7fd1780046c0 comp rx=0 tx=0).stop 2024-04-26T21:08:54.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.309+0000 7fd18d800640 1 -- 172.21.15.26:0/2073682030 shutdown_connections 2024-04-26T21:08:54.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.309+0000 7fd18d800640 1 --2- 172.21.15.26:0/2073682030 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd16406c8f0 0x7fd16406edb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:54.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.309+0000 7fd18d800640 1 --2- 172.21.15.26:0/2073682030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd18815fdd0 0x7fd18815a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:54.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.309+0000 7fd18d800640 1 --2- 172.21.15.26:0/2073682030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd188159b50 0x7fd1881559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:54.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.309+0000 7fd18d800640 1 --2- 172.21.15.26:0/2073682030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd188159770 0x7fd18815ea60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:54.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.309+0000 7fd18d800640 1 -- 172.21.15.26:0/2073682030 >> 172.21.15.26:0/2073682030 conn(0x7fd18806c690 msgr2=0x7fd188073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:54.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.309+0000 7fd18d800640 1 -- 172.21.15.26:0/2073682030 shutdown_connections 2024-04-26T21:08:54.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.309+0000 7fd18d800640 1 -- 172.21.15.26:0/2073682030 wait complete. 2024-04-26T21:08:54.330 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:08:54.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.531+0000 7f4993000640 1 Processor -- start 2024-04-26T21:08:54.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.531+0000 7f4993000640 1 -- start start 2024-04-26T21:08:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.532+0000 7f4993000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f498c146990 0x7f498c0b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.533+0000 7f4993000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f498c0b4480 con 0x7f498c06ec20 2024-04-26T21:08:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.533+0000 7f4993000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f498c073b70 con 0x7f498c0d13e0 2024-04-26T21:08:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.533+0000 7f4993000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f498c146d70 con 0x7f498c146990 2024-04-26T21:08:54.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.533+0000 7f498b400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f498c06ec20 0x7f498c0d0cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55182/0 (socket says 172.21.15.26:55182) 2024-04-26T21:08:54.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.533+0000 7f498b400640 1 -- 172.21.15.26:0/73583771 learned_addr learned my addr 172.21.15.26:0/73583771 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:08:54.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.533+0000 7f498aa00640 1 -- 172.21.15.26:0/73583771 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 356418663 0 0) 0x7f498c0b4480 con 0x7f498c06ec20 2024-04-26T21:08:54.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.533+0000 7f498aa00640 1 -- 172.21.15.26:0/73583771 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4970003520 con 0x7f498c06ec20 2024-04-26T21:08:54.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.533+0000 7f498be00640 1 --2- 172.21.15.26:0/73583771 >> v2:172.21.15.86:3300/0 conn(0x7f498c146990 0x7f498c0b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:54.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.534+0000 7f498aa00640 1 -- 172.21.15.26:0/73583771 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1143958791 0 0) 0x7f4970003520 con 0x7f498c06ec20 2024-04-26T21:08:54.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.534+0000 7f498aa00640 1 -- 172.21.15.26:0/73583771 >> v2:172.21.15.86:3300/0 conn(0x7f498c146990 msgr2=0x7f498c0b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:54.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.534+0000 7f498aa00640 1 --2- 172.21.15.26:0/73583771 >> v2:172.21.15.86:3300/0 conn(0x7f498c146990 0x7f498c0b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:54.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.534+0000 7f498aa00640 1 -- 172.21.15.26:0/73583771 >> v1:172.21.15.86:6789/0 conn(0x7f498c0d13e0 legacy=0x7f498c072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:54.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.534+0000 7f498aa00640 1 -- 172.21.15.26:0/73583771 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498c073e50 con 0x7f498c06ec20 2024-04-26T21:08:54.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.534+0000 7f498aa00640 1 -- 172.21.15.26:0/73583771 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f4974001670 con 0x7f498c06ec20 2024-04-26T21:08:54.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.534+0000 7f498be00640 1 --2- 172.21.15.26:0/73583771 >> v2:172.21.15.86:3300/0 conn(0x7f498c146990 0x7f498c0b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:08:54.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.534+0000 7f498aa00640 1 -- 172.21.15.26:0/73583771 >> v1:172.21.15.26:6789/0 conn(0x7f498c06ec20 legacy=0x7f498c0d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:54.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.534+0000 7f498aa00640 1 --2- 172.21.15.26:0/73583771 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4970003a00 0x7f4970003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:54.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.534+0000 7f498aa00640 1 --2- 172.21.15.26:0/73583771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4970004800 0x7f4970004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:54.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.534+0000 7f498aa00640 1 --2- 172.21.15.26:0/73583771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4970005210 0x7f497000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:54.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.534+0000 7f498aa00640 1 -- 172.21.15.26:0/73583771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f498c146d70 con 0x7f4970004800 2024-04-26T21:08:54.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.534+0000 7f498aa00640 1 -- 172.21.15.26:0/73583771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f497000c160 con 0x7f4970005210 2024-04-26T21:08:54.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.534+0000 7f498aa00640 1 -- 172.21.15.26:0/73583771 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f497000c360 con 0x7f4970003a00 2024-04-26T21:08:54.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.534+0000 7f498b400640 1 --2- 172.21.15.26:0/73583771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4970004800 0x7f4970004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.534+0000 7f498be00640 1 --2- 172.21.15.26:0/73583771 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4970003a00 0x7f4970003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.535+0000 7f498b400640 1 -- 172.21.15.26:0/73583771 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4970003a00 msgr2=0x7f4970003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.535+0000 7f498b400640 1 --2- 172.21.15.26:0/73583771 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4970003a00 0x7f4970003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.535+0000 7f4991000640 1 --2- 172.21.15.26:0/73583771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4970005210 0x7f497000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.535+0000 7f498b400640 1 -- 172.21.15.26:0/73583771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4970005210 msgr2=0x7f497000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.535+0000 7f498b400640 1 --2- 172.21.15.26:0/73583771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4970005210 0x7f497000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.535+0000 7f498b400640 1 -- 172.21.15.26:0/73583771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4970003520 con 0x7f4970004800 2024-04-26T21:08:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.535+0000 7f4991000640 1 -- 172.21.15.26:0/73583771 reap_dead start 2024-04-26T21:08:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.536+0000 7f498b400640 1 --2- 172.21.15.26:0/73583771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4970004800 0x7f4970004c80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f497400c0e0 tx=0x7f4970005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fdb97a53413f915 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.536+0000 7f498aa00640 1 -- 172.21.15.26:0/73583771 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974007c20 con 0x7f4970004800 2024-04-26T21:08:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.536+0000 7f498aa00640 1 -- 172.21.15.26:0/73583771 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4974005af0 con 0x7f4970004800 2024-04-26T21:08:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.536+0000 7f498aa00640 1 -- 172.21.15.26:0/73583771 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974007070 con 0x7f4970004800 2024-04-26T21:08:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.536+0000 7f4993000640 1 -- 172.21.15.26:0/73583771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4970004800 msgr2=0x7f4970004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:54.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.536+0000 7f4993000640 1 --2- 172.21.15.26:0/73583771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4970004800 0x7f4970004c80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f497400c0e0 tx=0x7f4970005830 comp rx=0 tx=0).stop 2024-04-26T21:08:54.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.536+0000 7f4993000640 1 -- 172.21.15.26:0/73583771 shutdown_connections 2024-04-26T21:08:54.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.536+0000 7f4993000640 1 --2- 172.21.15.26:0/73583771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4970004800 0x7f4970004c80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:54.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.536+0000 7f4993000640 1 -- 172.21.15.26:0/73583771 >> 172.21.15.26:0/73583771 conn(0x7f498c0cd840 msgr2=0x7f498c0cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:54.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.536+0000 7f4993000640 1 -- 172.21.15.26:0/73583771 shutdown_connections 2024-04-26T21:08:54.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.537+0000 7f4993000640 1 -- 172.21.15.26:0/73583771 wait complete. 2024-04-26T21:08:54.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.538+0000 7f4993000640 1 Processor -- start 2024-04-26T21:08:54.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.538+0000 7f4993000640 1 -- start start 2024-04-26T21:08:54.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.539+0000 7f4993000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4970004800 0x7f498c0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:54.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.539+0000 7f4993000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498c15c180 0x7f498c15c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:54.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.539+0000 7f4993000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f498c0af940 0x7f498c0afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:54.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.539+0000 7f4993000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f498c06e700 con 0x7f498c15c180 2024-04-26T21:08:54.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.539+0000 7f4993000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f498c06e420 con 0x7f4970004800 2024-04-26T21:08:54.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.539+0000 7f4993000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f498c06f0b0 con 0x7f498c0af940 2024-04-26T21:08:54.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.539+0000 7f498b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498c15c180 0x7f498c15c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:54.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.540+0000 7f498b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498c15c180 0x7f498c15c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34934/0 (socket says 172.21.15.26:34934) 2024-04-26T21:08:54.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.540+0000 7f498b400640 1 -- 172.21.15.26:0/2516660479 learned_addr learned my addr 172.21.15.26:0/2516660479 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:54.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.540+0000 7f498be00640 1 --2- 172.21.15.26:0/2516660479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4970004800 0x7f498c0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:54.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.540+0000 7f4991000640 1 --2- 172.21.15.26:0/2516660479 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f498c0af940 0x7f498c0afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:54.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.540+0000 7f498b400640 1 -- 172.21.15.26:0/2516660479 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f498c0af940 msgr2=0x7f498c0afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:54.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.540+0000 7f498b400640 1 --2- 172.21.15.26:0/2516660479 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f498c0af940 0x7f498c0afdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:54.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.540+0000 7f498b400640 1 -- 172.21.15.26:0/2516660479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4970004800 msgr2=0x7f498c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:54.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.540+0000 7f498b400640 1 --2- 172.21.15.26:0/2516660479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4970004800 0x7f498c0d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:54.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.540+0000 7f498b400640 1 -- 172.21.15.26:0/2516660479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498c0cf1e0 con 0x7f498c15c180 2024-04-26T21:08:54.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.540+0000 7f498b400640 1 --2- 172.21.15.26:0/2516660479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498c15c180 0x7f498c15c5e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f497400c200 tx=0x7f498c073d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:54.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.541+0000 7f4983e00640 1 -- 172.21.15.26:0/2516660479 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974041020 con 0x7f498c15c180 2024-04-26T21:08:54.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.541+0000 7f4983e00640 1 -- 172.21.15.26:0/2516660479 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4974038040 con 0x7f498c15c180 2024-04-26T21:08:54.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.541+0000 7f4983e00640 1 -- 172.21.15.26:0/2516660479 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974001670 con 0x7f498c15c180 2024-04-26T21:08:54.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.541+0000 7f4993000640 1 -- 172.21.15.26:0/2516660479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498c0cf4c0 con 0x7f498c15c180 2024-04-26T21:08:54.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.541+0000 7f4993000640 1 -- 172.21.15.26:0/2516660479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f498c0cfd60 con 0x7f498c15c180 2024-04-26T21:08:54.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.542+0000 7f4993000640 1 -- 172.21.15.26:0/2516660479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f498c0cf8b0 con 0x7f498c15c180 2024-04-26T21:08:54.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.543+0000 7f4983e00640 1 -- 172.21.15.26:0/2516660479 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4974044020 con 0x7f498c15c180 2024-04-26T21:08:54.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.544+0000 7f4983e00640 1 --2- 172.21.15.26:0/2516660479 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f496806c920 0x7f496806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:54.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.544+0000 7f4983e00640 1 -- 172.21.15.26:0/2516660479 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f49740abe80 con 0x7f498c15c180 2024-04-26T21:08:54.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.544+0000 7f498be00640 1 --2- 172.21.15.26:0/2516660479 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f496806c920 0x7f496806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:54.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.545+0000 7f498be00640 1 --2- 172.21.15.26:0/2516660479 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f496806c920 0x7f496806ede0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f497c003fd0 tx=0x7f497c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:54.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.548+0000 7f4983e00640 1 -- 172.21.15.26:0/2516660479 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f497407a340 con 0x7f498c15c180 2024-04-26T21:08:54.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.792+0000 7f4993000640 1 -- 172.21.15.26:0/2516660479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f498c0cf690 con 0x7f498c15c180 2024-04-26T21:08:54.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.793+0000 7f4983e00640 1 -- 172.21.15.26:0/2516660479 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7f4974079c40 con 0x7f498c15c180 2024-04-26T21:08:54.795 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:08:54.795 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/2695371801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2695371801},{"type":"v1","addr":"172.21.15.26:6836","nonce":2695371801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3349732696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3349732696},{"type":"v1","addr":"172.21.15.26:6837","nonce":3349732696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/2288633870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2288633870},{"type":"v1","addr":"172.21.15.86:6833","nonce":2288633870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:08:37.622844+0000","modified":"2024-04-26T21:08:40.715140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/2125360615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2125360615},{"type":"v1","addr":"172.21.15.86:6835","nonce":2125360615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4261,"qdb_cluster":[4261]},"id":1}]} 2024-04-26T21:08:54.796 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2024-04-26T21:08:54.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.796+0000 7f4993000640 1 -- 172.21.15.26:0/2516660479 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f496806c920 msgr2=0x7f496806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:54.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.796+0000 7f4993000640 1 --2- 172.21.15.26:0/2516660479 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f496806c920 0x7f496806ede0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f497c003fd0 tx=0x7f497c00b040 comp rx=0 tx=0).stop 2024-04-26T21:08:54.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.797+0000 7f4993000640 1 -- 172.21.15.26:0/2516660479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498c15c180 msgr2=0x7f498c15c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:54.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.797+0000 7f4993000640 1 --2- 172.21.15.26:0/2516660479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498c15c180 0x7f498c15c5e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f497400c200 tx=0x7f498c073d40 comp rx=0 tx=0).stop 2024-04-26T21:08:54.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.797+0000 7f4993000640 1 -- 172.21.15.26:0/2516660479 shutdown_connections 2024-04-26T21:08:54.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.797+0000 7f4993000640 1 --2- 172.21.15.26:0/2516660479 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f496806c920 0x7f496806ede0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:54.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.797+0000 7f4993000640 1 --2- 172.21.15.26:0/2516660479 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f498c0af940 0x7f498c0afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:54.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.797+0000 7f4993000640 1 --2- 172.21.15.26:0/2516660479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498c15c180 0x7f498c15c5e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:54.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.797+0000 7f4993000640 1 --2- 172.21.15.26:0/2516660479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4970004800 0x7f498c0d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:54.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.797+0000 7f4993000640 1 -- 172.21.15.26:0/2516660479 >> 172.21.15.26:0/2516660479 conn(0x7f498c0cd840 msgr2=0x7f498c156ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:54.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.797+0000 7f4993000640 1 -- 172.21.15.26:0/2516660479 shutdown_connections 2024-04-26T21:08:54.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:54.797+0000 7f4993000640 1 -- 172.21.15.26:0/2516660479 wait complete. 2024-04-26T21:08:55.221 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:08:37.622844+0000', 'modified': '2024-04-26T21:08:40.715140+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4261}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4261': {'gid': 4261, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6835/2125360615', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 2125360615}, {'type': 'v1', 'addr': '172.21.15.86:6835', 'nonce': 2125360615}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4261, 'qdb_cluster': [4261]} 2024-04-26T21:08:55.221 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:08:55.221 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-26T21:08:55.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.437+0000 7fb13b200640 1 Processor -- start 2024-04-26T21:08:55.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.438+0000 7fb13b200640 1 -- start start 2024-04-26T21:08:55.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.439+0000 7fb13b200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb13c145c20 0x7fb13c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.439+0000 7fb13b200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb13c0cf690 con 0x7fb13c146450 2024-04-26T21:08:55.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.439+0000 7fb13b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb13c0cf930 con 0x7fb13c0d1840 2024-04-26T21:08:55.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.439+0000 7fb13b200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fb13c0ced10 con 0x7fb13c145c20 2024-04-26T21:08:55.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.440+0000 7fb139e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb13c145c20 0x7fb13c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:55.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.440+0000 7fb139e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb13c145c20 0x7fb13c0d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34938/0 (socket says 172.21.15.26:34938) 2024-04-26T21:08:55.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.440+0000 7fb139e00640 1 -- 172.21.15.26:0/4178457542 learned_addr learned my addr 172.21.15.26:0/4178457542 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:55.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.440+0000 7fb139e00640 1 -- 172.21.15.26:0/4178457542 >> v1:172.21.15.86:6789/0 conn(0x7fb13c0d1840 legacy=0x7fb13c071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:55.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.440+0000 7fb139e00640 1 -- 172.21.15.26:0/4178457542 >> v1:172.21.15.26:6789/0 conn(0x7fb13c146450 legacy=0x7fb13c0d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:55.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.440+0000 7fb139e00640 1 -- 172.21.15.26:0/4178457542 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb13c0cfc10 con 0x7fb13c145c20 2024-04-26T21:08:55.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.441+0000 7fb139e00640 1 --2- 172.21.15.26:0/4178457542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb13c145c20 0x7fb13c0d06f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb12c009870 tx=0x7fb12c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa75de18b6d9484 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:55.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.442+0000 7fb139400640 1 -- 172.21.15.26:0/4178457542 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c037070 con 0x7fb13c145c20 2024-04-26T21:08:55.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.442+0000 7fb139400640 1 -- 172.21.15.26:0/4178457542 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb12c032440 con 0x7fb13c145c20 2024-04-26T21:08:55.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.442+0000 7fb139400640 1 -- 172.21.15.26:0/4178457542 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c032810 con 0x7fb13c145c20 2024-04-26T21:08:55.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.442+0000 7fb13b200640 1 -- 172.21.15.26:0/4178457542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb13c145c20 msgr2=0x7fb13c0d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:55.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.442+0000 7fb13b200640 1 --2- 172.21.15.26:0/4178457542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb13c145c20 0x7fb13c0d06f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb12c009870 tx=0x7fb12c02a2a0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.442+0000 7fb13b200640 1 -- 172.21.15.26:0/4178457542 shutdown_connections 2024-04-26T21:08:55.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.442+0000 7fb13b200640 1 --2- 172.21.15.26:0/4178457542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb13c145c20 0x7fb13c0d06f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.443+0000 7fb13b200640 1 -- 172.21.15.26:0/4178457542 >> 172.21.15.26:0/4178457542 conn(0x7fb13c06c690 msgr2=0x7fb13c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:55.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.443+0000 7fb13b200640 1 -- 172.21.15.26:0/4178457542 shutdown_connections 2024-04-26T21:08:55.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.443+0000 7fb13b200640 1 -- 172.21.15.26:0/4178457542 wait complete. 2024-04-26T21:08:55.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.445+0000 7fb13b200640 1 Processor -- start 2024-04-26T21:08:55.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.445+0000 7fb13b200640 1 -- start start 2024-04-26T21:08:55.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.446+0000 7fb13b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb13c0d1840 0x7fb13c156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.446+0000 7fb13b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb13c145c20 0x7fb13c159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.446+0000 7fb13b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c146450 0x7fb13c159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.446+0000 7fb13b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c146000 con 0x7fb13c0d1840 2024-04-26T21:08:55.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.446+0000 7fb13b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c0d01d0 con 0x7fb13c146450 2024-04-26T21:08:55.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.446+0000 7fb13b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb13c0b3ee0 con 0x7fb13c145c20 2024-04-26T21:08:55.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.446+0000 7fb139e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb13c0d1840 0x7fb13c156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:55.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.446+0000 7fb139e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb13c0d1840 0x7fb13c156d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34940/0 (socket says 172.21.15.26:34940) 2024-04-26T21:08:55.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.446+0000 7fb139e00640 1 -- 172.21.15.26:0/2601658044 learned_addr learned my addr 172.21.15.26:0/2601658044 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.447+0000 7fb139e00640 1 -- 172.21.15.26:0/2601658044 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb13c145c20 msgr2=0x7fb13c159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.447+0000 7fb133e00640 1 --2- 172.21.15.26:0/2601658044 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb13c145c20 0x7fb13c159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.447+0000 7fb139e00640 1 --2- 172.21.15.26:0/2601658044 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb13c145c20 0x7fb13c159210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.447+0000 7fb139e00640 1 -- 172.21.15.26:0/2601658044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c146450 msgr2=0x7fb13c159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.447+0000 7fb13a800640 1 --2- 172.21.15.26:0/2601658044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c146450 0x7fb13c159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.447+0000 7fb139e00640 1 --2- 172.21.15.26:0/2601658044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c146450 0x7fb13c159750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.447+0000 7fb139e00640 1 -- 172.21.15.26:0/2601658044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb13c0b41c0 con 0x7fb13c0d1840 2024-04-26T21:08:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.447+0000 7fb139e00640 1 --2- 172.21.15.26:0/2601658044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb13c0d1840 0x7fb13c156d30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb12c009870 tx=0x7fb12c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.448+0000 7fb132000640 1 -- 172.21.15.26:0/2601658044 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c041020 con 0x7fb13c0d1840 2024-04-26T21:08:55.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.448+0000 7fb132000640 1 -- 172.21.15.26:0/2601658044 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb12c030970 con 0x7fb13c0d1840 2024-04-26T21:08:55.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.448+0000 7fb132000640 1 -- 172.21.15.26:0/2601658044 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c037040 con 0x7fb13c0d1840 2024-04-26T21:08:55.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.448+0000 7fb13b200640 1 -- 172.21.15.26:0/2601658044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb13c0b44a0 con 0x7fb13c0d1840 2024-04-26T21:08:55.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.448+0000 7fb13b200640 1 -- 172.21.15.26:0/2601658044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb13c0b4a60 con 0x7fb13c0d1840 2024-04-26T21:08:55.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.450+0000 7fb132000640 1 -- 172.21.15.26:0/2601658044 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb12c044020 con 0x7fb13c0d1840 2024-04-26T21:08:55.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.451+0000 7fb132000640 1 --2- 172.21.15.26:0/2601658044 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb11806c940 0x7fb11806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.452+0000 7fb132000640 1 -- 172.21.15.26:0/2601658044 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb12c0ac500 con 0x7fb13c0d1840 2024-04-26T21:08:55.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.452+0000 7fb1437f2740 1 Processor -- start 2024-04-26T21:08:55.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.452+0000 7fb133e00640 1 --2- 172.21.15.26:0/2601658044 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb11806c940 0x7fb11806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:55.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.452+0000 7fb1437f2740 1 -- start start 2024-04-26T21:08:55.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.453+0000 7fb133e00640 1 --2- 172.21.15.26:0/2601658044 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb11806c940 0x7fb11806ee00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb13c15f670 tx=0x7fb124009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:55.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.453+0000 7fb1437f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2255877b0 0x55d225589910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.453+0000 7fb1437f2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255c3e80 0x55d225587b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.453+0000 7fb1437f2740 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55d2255b2270 0x55d2255b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.453+0000 7fb1437f2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d2254eb600 con 0x55d2255c3e80 2024-04-26T21:08:55.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.453+0000 7fb1437f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d225488100 con 0x55d2255877b0 2024-04-26T21:08:55.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.453+0000 7fb1437f2740 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55d2254e75c0 con 0x55d2255b2270 2024-04-26T21:08:55.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.453+0000 7fb133e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255c3e80 0x55d225587b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:55.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.453+0000 7fb133e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255c3e80 0x55d225587b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34950/0 (socket says 172.21.15.26:34950) 2024-04-26T21:08:55.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.453+0000 7fb133e00640 1 -- 172.21.15.26:0/2890715859 learned_addr learned my addr 172.21.15.26:0/2890715859 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:55.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.454+0000 7fb139e00640 1 --2- 172.21.15.26:0/2890715859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2255877b0 0x55d225589910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:55.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.454+0000 7fb13a800640 1 --2- 172.21.15.26:0/2890715859 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55d2255b2270 0x55d2255b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:55.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.454+0000 7fb133e00640 1 -- 172.21.15.26:0/2890715859 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55d2255b2270 msgr2=0x55d2255b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:55.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.454+0000 7fb133e00640 1 --2- 172.21.15.26:0/2890715859 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55d2255b2270 0x55d2255b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.454+0000 7fb133e00640 1 -- 172.21.15.26:0/2890715859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2255877b0 msgr2=0x55d225589910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:55.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.454+0000 7fb133e00640 1 --2- 172.21.15.26:0/2890715859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2255877b0 0x55d225589910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.454+0000 7fb133e00640 1 -- 172.21.15.26:0/2890715859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2251fd2d0 con 0x55d2255c3e80 2024-04-26T21:08:55.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.454+0000 7fb133e00640 1 --2- 172.21.15.26:0/2890715859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255c3e80 0x55d225587b90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb124014720 tx=0x7fb124010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee31839fb0ae7111 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:55.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.455+0000 7fb12a600640 1 -- 172.21.15.26:0/2890715859 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124024070 con 0x55d2255c3e80 2024-04-26T21:08:55.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.455+0000 7fb12a600640 1 -- 172.21.15.26:0/2890715859 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb124012a20 con 0x55d2255c3e80 2024-04-26T21:08:55.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.455+0000 7fb12a600640 1 -- 172.21.15.26:0/2890715859 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124012dd0 con 0x55d2255c3e80 2024-04-26T21:08:55.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.455+0000 7fb1437f2740 1 -- 172.21.15.26:0/2890715859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255c3e80 msgr2=0x55d225587b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:55.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.455+0000 7fb1437f2740 1 --2- 172.21.15.26:0/2890715859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255c3e80 0x55d225587b90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb124014720 tx=0x7fb124010000 comp rx=0 tx=0).stop 2024-04-26T21:08:55.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.455+0000 7fb1437f2740 1 -- 172.21.15.26:0/2890715859 shutdown_connections 2024-04-26T21:08:55.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.455+0000 7fb1437f2740 1 --2- 172.21.15.26:0/2890715859 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55d2255b2270 0x55d2255b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.455+0000 7fb1437f2740 1 --2- 172.21.15.26:0/2890715859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255c3e80 0x55d225587b90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.455+0000 7fb1437f2740 1 --2- 172.21.15.26:0/2890715859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2255877b0 0x55d225589910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.455+0000 7fb1437f2740 1 -- 172.21.15.26:0/2890715859 >> 172.21.15.26:0/2890715859 conn(0x55d22551bd10 msgr2=0x55d2255cd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:55.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.456+0000 7fb1437f2740 1 -- 172.21.15.26:0/2890715859 shutdown_connections 2024-04-26T21:08:55.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.456+0000 7fb1437f2740 1 -- 172.21.15.26:0/2890715859 wait complete. 2024-04-26T21:08:55.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.456+0000 7fb1437f2740 1 -- start start 2024-04-26T21:08:55.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.457+0000 7fb1437f2740 1 Processor -- start 2024-04-26T21:08:55.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.458+0000 7fb1437f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2255877b0 0x55d2255c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.458+0000 7fb1437f2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255b2270 0x55d2255dd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.458+0000 7fb1437f2740 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55d2255c3e80 0x55d2255b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.458+0000 7fb1437f2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d225472f70 con 0x55d2255b2270 2024-04-26T21:08:55.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.458+0000 7fb1437f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d2255afdc0 con 0x55d2255877b0 2024-04-26T21:08:55.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.458+0000 7fb1437f2740 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55d2255b46d0 con 0x55d2255c3e80 2024-04-26T21:08:55.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.458+0000 7fb1437f2740 10 client.0 _ll_register_callbacks cb 0x55d2254e1480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:08:55.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.458+0000 7fb133e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255b2270 0x55d2255dd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:55.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.458+0000 7fb133e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255b2270 0x55d2255dd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34960/0 (socket says 172.21.15.26:34960) 2024-04-26T21:08:55.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.458+0000 7fb133e00640 1 -- 172.21.15.26:0/32863621 learned_addr learned my addr 172.21.15.26:0/32863621 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:55.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.458+0000 7fb139e00640 1 --2- 172.21.15.26:0/32863621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2255877b0 0x55d2255c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:55.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.458+0000 7fb133e00640 1 -- 172.21.15.26:0/32863621 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55d2255c3e80 msgr2=0x55d2255b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:55.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.458+0000 7fb133e00640 1 --2- 172.21.15.26:0/32863621 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55d2255c3e80 0x55d2255b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.458+0000 7fb133e00640 1 -- 172.21.15.26:0/32863621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2255877b0 msgr2=0x55d2255c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:55.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.458+0000 7fb133e00640 1 --2- 172.21.15.26:0/32863621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2255877b0 0x55d2255c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.458+0000 7fb133e00640 1 -- 172.21.15.26:0/32863621 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2252c76b0 con 0x55d2255b2270 2024-04-26T21:08:55.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.458+0000 7fb139e00640 1 --2- 172.21.15.26:0/32863621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2255877b0 0x55d2255c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:55.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.459+0000 7fb133e00640 1 --2- 172.21.15.26:0/32863621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255b2270 0x55d2255dd480 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb124022040 tx=0x7fb124000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96cef1227d38c1b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:55.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.459+0000 7fb11ea00640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-26T21:08:55.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.459+0000 7fb11ea00640 1 -- 172.21.15.26:0/32863621 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124024040 con 0x55d2255b2270 2024-04-26T21:08:55.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.459+0000 7fb11ea00640 1 -- 172.21.15.26:0/32863621 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb124029460 con 0x55d2255b2270 2024-04-26T21:08:55.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.459+0000 7fb1437f2740 1 -- 172.21.15.26:0/32863621 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb124012a20 con 0x55d2255b2270 2024-04-26T21:08:55.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.459+0000 7fb11ea00640 1 -- 172.21.15.26:0/32863621 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fb124038050 con 0x55d2255b2270 2024-04-26T21:08:55.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.459+0000 7fb1437f2740 10 client.4647 fetch_fsmap learned FSMap version 8 2024-04-26T21:08:55.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.459+0000 7fb1437f2740 1 -- 172.21.15.26:0/32863621 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55d2255b4190 con 0x55d2255b2270 2024-04-26T21:08:55.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.459+0000 7fb11ea00640 1 -- 172.21.15.26:0/32863621 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb124012a20 con 0x55d2255b2270 2024-04-26T21:08:55.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.460+0000 7fb11ea00640 1 -- 172.21.15.26:0/32863621 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fb12403b020 con 0x55d2255b2270 2024-04-26T21:08:55.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.460+0000 7fb1437f2740 10 client.4647 fetch_fsmap finished waiting for FSMap version 8 2024-04-26T21:08:55.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.460+0000 7fb1437f2740 10 client.4647 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-26T21:08:55.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.460+0000 7fb1437f2740 20 client.4647 populate_metadata read hostname 'smithi026' 2024-04-26T21:08:55.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.460+0000 7fb1437f2740 1 --2- 172.21.15.26:0/32863621 >> [v2:172.21.15.86:6834/2125360615,v1:172.21.15.86:6835/2125360615] conn(0x55d2255c7810 0x55d2255e0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.460+0000 7fb1437f2740 4 client.4647 mds_command: new command op to 4261 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:08:55.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.460+0000 7fb1437f2740 1 -- 172.21.15.26:0/32863621 --> [v2:172.21.15.86:6834/2125360615,v1:172.21.15.86:6835/2125360615] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2254f9210 con 0x55d2255c7810 2024-04-26T21:08:55.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.460+0000 7fb139e00640 1 --2- 172.21.15.26:0/32863621 >> [v2:172.21.15.86:6834/2125360615,v1:172.21.15.86:6835/2125360615] conn(0x55d2255c7810 0x55d2255e0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:55.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.461+0000 7fb139e00640 1 --2- 172.21.15.26:0/32863621 >> [v2:172.21.15.86:6834/2125360615,v1:172.21.15.86:6835/2125360615] conn(0x55d2255c7810 0x55d2255e0c30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e0462124b762e11 server_cookie=aafd87f129dde53 in_seq=0 out_seq=0 2024-04-26T21:08:55.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.461+0000 7fb11ea00640 10 client.4647 ms_handle_connect on v2:172.21.15.86:6834/2125360615 2024-04-26T21:08:55.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.463+0000 7fb11ea00640 1 -- 172.21.15.26:0/32863621 <== mds.0 v2:172.21.15.86:6834/2125360615 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2254f9210 con 0x55d2255c7810 2024-04-26T21:08:55.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.463+0000 7fb11ea00640 10 client.4647 handle_command_reply: tid=0 2024-04-26T21:08:55.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.463+0000 7fb1437f2740 1 client.4647 shutdown 2024-04-26T21:08:55.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.464+0000 7fb1437f2740 1 -- 172.21.15.26:0/32863621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255b2270 msgr2=0x55d2255dd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:55.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.464+0000 7fb1437f2740 1 --2- 172.21.15.26:0/32863621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255b2270 0x55d2255dd480 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb124022040 tx=0x7fb124000bc0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.464+0000 7fb1437f2740 1 -- 172.21.15.26:0/32863621 shutdown_connections 2024-04-26T21:08:55.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.464+0000 7fb1437f2740 1 --2- 172.21.15.26:0/32863621 >> [v2:172.21.15.86:6834/2125360615,v1:172.21.15.86:6835/2125360615] conn(0x55d2255c7810 0x55d2255e0c30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.464+0000 7fb1437f2740 1 --2- 172.21.15.26:0/32863621 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55d2255c3e80 0x55d2255b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.464+0000 7fb1437f2740 1 --2- 172.21.15.26:0/32863621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255b2270 0x55d2255dd480 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.464+0000 7fb1437f2740 1 --2- 172.21.15.26:0/32863621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2255877b0 0x55d2255c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.464+0000 7fb11ea00640 0 client.4647 ms_handle_reset on v2:172.21.15.86:6834/2125360615 2024-04-26T21:08:55.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.464+0000 7fb1437f2740 1 -- 172.21.15.26:0/32863621 >> 172.21.15.26:0/32863621 conn(0x55d22551bd10 msgr2=0x55d2255b01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:55.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.464+0000 7fb1437f2740 1 -- 172.21.15.26:0/32863621 shutdown_connections 2024-04-26T21:08:55.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.464+0000 7fb1437f2740 1 -- 172.21.15.26:0/32863621 wait complete. 2024-04-26T21:08:55.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.464+0000 7fb1437f2740 20 client.4647 trim_cache size 0 max 16384 2024-04-26T21:08:55.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.484+0000 7fb1437f2740 1 Processor -- start 2024-04-26T21:08:55.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.484+0000 7fb1437f2740 1 -- start start 2024-04-26T21:08:55.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.485+0000 7fb1437f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2255877b0 0x55d2255affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.485+0000 7fb1437f2740 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55d2255b2270 0x55d2255e6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.485+0000 7fb1437f2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255c3e80 0x55d2255e73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.485+0000 7fb1437f2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c0b5d40 con 0x55d2255c3e80 2024-04-26T21:08:55.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.485+0000 7fb1437f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c0b50a0 con 0x55d2255877b0 2024-04-26T21:08:55.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.485+0000 7fb1437f2740 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb13c0b56f0 con 0x55d2255b2270 2024-04-26T21:08:55.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.486+0000 7fb13a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255c3e80 0x55d2255e73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:55.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.486+0000 7fb13a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255c3e80 0x55d2255e73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34970/0 (socket says 172.21.15.26:34970) 2024-04-26T21:08:55.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.486+0000 7fb13a800640 1 -- 172.21.15.26:0/2076438806 learned_addr learned my addr 172.21.15.26:0/2076438806 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:55.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.486+0000 7fb133e00640 1 --2- 172.21.15.26:0/2076438806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55d2255b2270 0x55d2255e6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:55.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.486+0000 7fb139e00640 1 --2- 172.21.15.26:0/2076438806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2255877b0 0x55d2255affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:55.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.486+0000 7fb13a800640 1 -- 172.21.15.26:0/2076438806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55d2255b2270 msgr2=0x55d2255e6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:55.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.486+0000 7fb13a800640 1 --2- 172.21.15.26:0/2076438806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55d2255b2270 0x55d2255e6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.486+0000 7fb13a800640 1 -- 172.21.15.26:0/2076438806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2255877b0 msgr2=0x55d2255affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:55.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.486+0000 7fb13a800640 1 --2- 172.21.15.26:0/2076438806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2255877b0 0x55d2255affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.486+0000 7fb13a800640 1 -- 172.21.15.26:0/2076438806 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2255c6e50 con 0x55d2255c3e80 2024-04-26T21:08:55.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.486+0000 7fb133e00640 1 --2- 172.21.15.26:0/2076438806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55d2255b2270 0x55d2255e6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:55.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.486+0000 7fb13a800640 1 --2- 172.21.15.26:0/2076438806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255c3e80 0x55d2255e73c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb134005390 tx=0x7fb134005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e6f48f21e8dda6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:55.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.487+0000 7fb129c00640 1 -- 172.21.15.26:0/2076438806 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb134018020 con 0x55d2255c3e80 2024-04-26T21:08:55.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.487+0000 7fb129c00640 1 -- 172.21.15.26:0/2076438806 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb13400fae0 con 0x55d2255c3e80 2024-04-26T21:08:55.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.487+0000 7fb129c00640 1 -- 172.21.15.26:0/2076438806 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb134014450 con 0x55d2255c3e80 2024-04-26T21:08:55.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.487+0000 7fb1437f2740 1 -- 172.21.15.26:0/2076438806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255c3e80 msgr2=0x55d2255e73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:55.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.487+0000 7fb1437f2740 1 --2- 172.21.15.26:0/2076438806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255c3e80 0x55d2255e73c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb134005390 tx=0x7fb134005850 comp rx=0 tx=0).stop 2024-04-26T21:08:55.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.487+0000 7fb1437f2740 1 -- 172.21.15.26:0/2076438806 shutdown_connections 2024-04-26T21:08:55.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.487+0000 7fb1437f2740 1 --2- 172.21.15.26:0/2076438806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255c3e80 0x55d2255e73c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.487+0000 7fb1437f2740 1 --2- 172.21.15.26:0/2076438806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55d2255b2270 0x55d2255e6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.487+0000 7fb1437f2740 1 --2- 172.21.15.26:0/2076438806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2255877b0 0x55d2255affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.487+0000 7fb1437f2740 1 -- 172.21.15.26:0/2076438806 >> 172.21.15.26:0/2076438806 conn(0x55d22551bd10 msgr2=0x55d22557ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:55.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.487+0000 7fb1437f2740 1 -- 172.21.15.26:0/2076438806 shutdown_connections 2024-04-26T21:08:55.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.487+0000 7fb1437f2740 1 -- 172.21.15.26:0/2076438806 wait complete. 2024-04-26T21:08:55.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.488+0000 7fb1437f2740 1 -- start start 2024-04-26T21:08:55.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.488+0000 7fb1437f2740 1 Processor -- start 2024-04-26T21:08:55.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.489+0000 7fb1437f2740 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55d2255877b0 0x55d225638dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.489+0000 7fb1437f2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255b2270 0x55d2255d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.489+0000 7fb1437f2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2255c3e80 0x55d2255d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.489+0000 7fb1437f2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d2255b35f0 con 0x55d2255b2270 2024-04-26T21:08:55.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.489+0000 7fb1437f2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d2255b4460 con 0x55d2255c3e80 2024-04-26T21:08:55.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.489+0000 7fb1437f2740 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55d2255c6010 con 0x55d2255877b0 2024-04-26T21:08:55.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.489+0000 7fb1437f2740 10 client.0 _ll_register_callbacks cb 0x7fb124022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:08:55.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.490+0000 7fb133e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255b2270 0x55d2255d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:55.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.490+0000 7fb133e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255b2270 0x55d2255d05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34972/0 (socket says 172.21.15.26:34972) 2024-04-26T21:08:55.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.490+0000 7fb133e00640 1 -- 172.21.15.26:0/3729455060 learned_addr learned my addr 172.21.15.26:0/3729455060 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:55.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.490+0000 7fb13a800640 1 --2- 172.21.15.26:0/3729455060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2255c3e80 0x55d2255d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:55.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.490+0000 7fb139e00640 1 --2- 172.21.15.26:0/3729455060 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55d2255877b0 0x55d225638dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:55.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.490+0000 7fb133e00640 1 -- 172.21.15.26:0/3729455060 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55d2255877b0 msgr2=0x55d225638dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:55.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.490+0000 7fb133e00640 1 --2- 172.21.15.26:0/3729455060 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55d2255877b0 0x55d225638dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.490+0000 7fb133e00640 1 -- 172.21.15.26:0/3729455060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2255c3e80 msgr2=0x55d2255d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.490+0000 7fb133e00640 1 --2- 172.21.15.26:0/3729455060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2255c3e80 0x55d2255d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.490+0000 7fb133e00640 1 -- 172.21.15.26:0/3729455060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2255ae160 con 0x55d2255b2270 2024-04-26T21:08:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.490+0000 7fb139e00640 1 --2- 172.21.15.26:0/3729455060 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55d2255877b0 0x55d225638dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.490+0000 7fb13a800640 1 --2- 172.21.15.26:0/3729455060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2255c3e80 0x55d2255d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.490+0000 7fb133e00640 1 --2- 172.21.15.26:0/3729455060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255b2270 0x55d2255d05a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb124009010 tx=0x7fb12401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af019e466a92fe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.490+0000 7fb11ea00640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-26T21:08:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.490+0000 7fb11ea00640 1 -- 172.21.15.26:0/3729455060 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124041060 con 0x55d2255b2270 2024-04-26T21:08:55.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.491+0000 7fb11ea00640 1 -- 172.21.15.26:0/3729455060 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb124029bd0 con 0x55d2255b2270 2024-04-26T21:08:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.491+0000 7fb1437f2740 1 -- 172.21.15.26:0/3729455060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb13400fae0 con 0x55d2255b2270 2024-04-26T21:08:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.491+0000 7fb11ea00640 1 -- 172.21.15.26:0/3729455060 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fb124047050 con 0x55d2255b2270 2024-04-26T21:08:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.491+0000 7fb1437f2740 10 client.4653 fetch_fsmap learned FSMap version 8 2024-04-26T21:08:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.491+0000 7fb1437f2740 1 -- 172.21.15.26:0/3729455060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55d2255da470 con 0x55d2255b2270 2024-04-26T21:08:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.491+0000 7fb11ea00640 1 -- 172.21.15.26:0/3729455060 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb13400fae0 con 0x55d2255b2270 2024-04-26T21:08:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.491+0000 7fb11ea00640 1 -- 172.21.15.26:0/3729455060 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fb12404a020 con 0x55d2255b2270 2024-04-26T21:08:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.491+0000 7fb1437f2740 10 client.4653 fetch_fsmap finished waiting for FSMap version 8 2024-04-26T21:08:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.491+0000 7fb1437f2740 10 client.4653 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-26T21:08:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.491+0000 7fb1437f2740 20 client.4653 populate_metadata read hostname 'smithi026' 2024-04-26T21:08:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.491+0000 7fb1437f2740 1 --2- 172.21.15.26:0/3729455060 >> [v2:172.21.15.86:6834/2125360615,v1:172.21.15.86:6835/2125360615] conn(0x55d2255b1190 0x55d225653a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.491+0000 7fb1437f2740 4 client.4653 mds_command: new command op to 4261 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:08:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.491+0000 7fb1437f2740 1 -- 172.21.15.26:0/3729455060 --> [v2:172.21.15.86:6834/2125360615,v1:172.21.15.86:6835/2125360615] -- command(tid 0: {"prefix": "status"}) -- 0x55d2254f3080 con 0x55d2255b1190 2024-04-26T21:08:55.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.492+0000 7fb139e00640 1 --2- 172.21.15.26:0/3729455060 >> [v2:172.21.15.86:6834/2125360615,v1:172.21.15.86:6835/2125360615] conn(0x55d2255b1190 0x55d225653a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:55.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.493+0000 7fb139e00640 1 --2- 172.21.15.26:0/3729455060 >> [v2:172.21.15.86:6834/2125360615,v1:172.21.15.86:6835/2125360615] conn(0x55d2255b1190 0x55d225653a60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ca7003d3d67f682 server_cookie=8fbbe644b327fc47 in_seq=0 out_seq=0 2024-04-26T21:08:55.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.493+0000 7fb11ea00640 10 client.4653 ms_handle_connect on v2:172.21.15.86:6834/2125360615 2024-04-26T21:08:55.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.493+0000 7fb11ea00640 1 -- 172.21.15.26:0/3729455060 <== mds.0 v2:172.21.15.86:6834/2125360615 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55d2254f3080 con 0x55d2255b1190 2024-04-26T21:08:55.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.493+0000 7fb11ea00640 10 client.4653 handle_command_reply: tid=0 2024-04-26T21:08:55.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.493+0000 7fb1437f2740 1 client.4653 shutdown 2024-04-26T21:08:55.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.494+0000 7fb1437f2740 1 -- 172.21.15.26:0/3729455060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255b2270 msgr2=0x55d2255d05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:55.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.494+0000 7fb1437f2740 1 --2- 172.21.15.26:0/3729455060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255b2270 0x55d2255d05a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb124009010 tx=0x7fb12401d000 comp rx=0 tx=0).stop 2024-04-26T21:08:55.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.494+0000 7fb1437f2740 1 -- 172.21.15.26:0/3729455060 shutdown_connections 2024-04-26T21:08:55.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.494+0000 7fb1437f2740 1 --2- 172.21.15.26:0/3729455060 >> [v2:172.21.15.86:6834/2125360615,v1:172.21.15.86:6835/2125360615] conn(0x55d2255b1190 0x55d225653a60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.494+0000 7fb1437f2740 1 --2- 172.21.15.26:0/3729455060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d2255c3e80 0x55d2255d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.494+0000 7fb1437f2740 1 --2- 172.21.15.26:0/3729455060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d2255b2270 0x55d2255d05a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.494+0000 7fb1437f2740 1 --2- 172.21.15.26:0/3729455060 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55d2255877b0 0x55d225638dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.494+0000 7fb11ea00640 0 client.4653 ms_handle_reset on v2:172.21.15.86:6834/2125360615 2024-04-26T21:08:55.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.494+0000 7fb1437f2740 1 -- 172.21.15.26:0/3729455060 >> 172.21.15.26:0/3729455060 conn(0x55d22551bd10 msgr2=0x55d22557ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:55.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.494+0000 7fb1437f2740 1 -- 172.21.15.26:0/3729455060 shutdown_connections 2024-04-26T21:08:55.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.494+0000 7fb1437f2740 1 -- 172.21.15.26:0/3729455060 wait complete. 2024-04-26T21:08:55.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.494+0000 7fb1437f2740 20 client.4653 trim_cache size 0 max 16384 2024-04-26T21:08:55.499 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-26T21:08:55.499 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "eaf60df9-83a0-4283-ab2c-83d2c02a2067", 2024-04-26T21:08:55.499 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-04-26T21:08:55.499 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4261, 2024-04-26T21:08:55.499 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-26T21:08:55.500 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-26T21:08:55.500 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-26T21:08:55.500 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 17.306382404000001, 2024-04-26T21:08:55.500 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 8, 2024-04-26T21:08:55.500 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 21, 2024-04-26T21:08:55.500 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 21, 2024-04-26T21:08:55.500 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 25.088187474000001 2024-04-26T21:08:55.500 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-26T21:08:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.495+0000 7fb13b200640 1 -- 172.21.15.26:0/2601658044 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb11806c940 msgr2=0x7fb11806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.495+0000 7fb13b200640 1 --2- 172.21.15.26:0/2601658044 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb11806c940 0x7fb11806ee00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb13c15f670 tx=0x7fb124009040 comp rx=0 tx=0).stop 2024-04-26T21:08:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.495+0000 7fb13b200640 1 -- 172.21.15.26:0/2601658044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb13c0d1840 msgr2=0x7fb13c156d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.495+0000 7fb13b200640 1 --2- 172.21.15.26:0/2601658044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb13c0d1840 0x7fb13c156d30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb12c009870 tx=0x7fb12c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.495+0000 7fb13b200640 1 -- 172.21.15.26:0/2601658044 shutdown_connections 2024-04-26T21:08:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.495+0000 7fb13b200640 1 --2- 172.21.15.26:0/2601658044 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb11806c940 0x7fb11806ee00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.495+0000 7fb13b200640 1 --2- 172.21.15.26:0/2601658044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb13c146450 0x7fb13c159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.495+0000 7fb13b200640 1 --2- 172.21.15.26:0/2601658044 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb13c145c20 0x7fb13c159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.495+0000 7fb13b200640 1 --2- 172.21.15.26:0/2601658044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb13c0d1840 0x7fb13c156d30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:55.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.495+0000 7fb13b200640 1 -- 172.21.15.26:0/2601658044 >> 172.21.15.26:0/2601658044 conn(0x7fb13c06c690 msgr2=0x7fb13c06ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:55.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.495+0000 7fb13b200640 1 -- 172.21.15.26:0/2601658044 shutdown_connections 2024-04-26T21:08:55.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:55.495+0000 7fb13b200640 1 -- 172.21.15.26:0/2601658044 wait complete. 2024-04-26T21:08:55.513 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T21:08:55.523 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T21:08:55.523 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T21:08:55.524 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T21:08:55.524 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T21:08:55.562 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T21:08:55.562 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-26T21:08:55.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:08:55.563 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-04-26T21:08:55.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:08:55.622 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-04-26T21:08:55.718 INFO:teuthology.orchestra.run.smithi026.stderr:Cannot find device "ceph-brx" 2024-04-26T21:08:55.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:08:55.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:08:55.723 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-04-26T21:08:55.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:08:55.780 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-04-26T21:08:55.873 INFO:teuthology.orchestra.run.smithi026.stderr:Cannot find device "ceph-brx" 2024-04-26T21:08:55.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:08:55.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:08:55.877 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-26T21:08:55.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:08:55.938 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-26T21:08:56.025 INFO:teuthology.orchestra.run.smithi086.stderr:Cannot find device "ceph-brx" 2024-04-26T21:08:56.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:08:56.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:08:56.029 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-26T21:08:56.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:08:56.083 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-26T21:08:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:Cannot find device "ceph-brx" 2024-04-26T21:08:56.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:08:56.174 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-04-26T21:08:56.174 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:08:56.174 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:08:56.174 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:08:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.391+0000 7f8e02600640 1 Processor -- start 2024-04-26T21:08:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.391+0000 7f8e02600640 1 -- start start 2024-04-26T21:08:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.393+0000 7f8e02600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8dfc160930 0x7f8dfc0b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.393+0000 7f8e02600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8dfc0b3370 0x7f8dfc15f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.393+0000 7f8e02600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dfc162b20 con 0x7f8dfc162740 2024-04-26T21:08:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.393+0000 7f8e02600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f8dfc0cd750 con 0x7f8dfc160930 2024-04-26T21:08:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.393+0000 7f8e02600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8dfc0cda30 con 0x7f8dfc0b3370 2024-04-26T21:08:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.393+0000 7f8df3e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8dfc160930 0x7f8dfc0b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.393+0000 7f8df3e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8dfc160930 0x7f8dfc0b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34978/0 (socket says 172.21.15.26:34978) 2024-04-26T21:08:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.393+0000 7f8df3e00640 1 -- 172.21.15.26:0/4204575739 learned_addr learned my addr 172.21.15.26:0/4204575739 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:56.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.394+0000 7f8e01c00640 1 --2- 172.21.15.26:0/4204575739 >> v2:172.21.15.86:3300/0 conn(0x7f8dfc0b3370 0x7f8dfc15f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:56.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.394+0000 7f8df3e00640 1 -- 172.21.15.26:0/4204575739 >> v2:172.21.15.86:3300/0 conn(0x7f8dfc0b3370 msgr2=0x7f8dfc15f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:56.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.394+0000 7f8df3e00640 1 --2- 172.21.15.26:0/4204575739 >> v2:172.21.15.86:3300/0 conn(0x7f8dfc0b3370 0x7f8dfc15f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:56.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.394+0000 7f8df3e00640 1 -- 172.21.15.26:0/4204575739 >> v1:172.21.15.86:6789/0 conn(0x7f8dfc162740 legacy=0x7f8dfc06fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:08:56.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.394+0000 7f8df3e00640 1 -- 172.21.15.26:0/4204575739 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dfc0cdd10 con 0x7f8dfc160930 2024-04-26T21:08:56.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.395+0000 7f8df3e00640 1 --2- 172.21.15.26:0/4204575739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8dfc160930 0x7f8dfc0b2e30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc06f580 tx=0x7f8de402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8ec1f4628595c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:56.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.395+0000 7f8dfbe00640 1 -- 172.21.15.26:0/4204575739 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4037070 con 0x7f8dfc160930 2024-04-26T21:08:56.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.395+0000 7f8dfbe00640 1 -- 172.21.15.26:0/4204575739 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de4032440 con 0x7f8dfc160930 2024-04-26T21:08:56.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.395+0000 7f8dfbe00640 1 -- 172.21.15.26:0/4204575739 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4032810 con 0x7f8dfc160930 2024-04-26T21:08:56.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.396+0000 7f8e02600640 1 -- 172.21.15.26:0/4204575739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8dfc160930 msgr2=0x7f8dfc0b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:56.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.396+0000 7f8e02600640 1 --2- 172.21.15.26:0/4204575739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8dfc160930 0x7f8dfc0b2e30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc06f580 tx=0x7f8de402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:08:56.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.396+0000 7f8e02600640 1 -- 172.21.15.26:0/4204575739 shutdown_connections 2024-04-26T21:08:56.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.396+0000 7f8e02600640 1 --2- 172.21.15.26:0/4204575739 >> v2:172.21.15.86:3300/0 conn(0x7f8dfc0b3370 0x7f8dfc15f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:56.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.396+0000 7f8e02600640 1 --2- 172.21.15.26:0/4204575739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8dfc160930 0x7f8dfc0b2e30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:56.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.396+0000 7f8e02600640 1 -- 172.21.15.26:0/4204575739 >> 172.21.15.26:0/4204575739 conn(0x7f8dfc06c660 msgr2=0x7f8dfc06ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:56.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.396+0000 7f8e02600640 1 -- 172.21.15.26:0/4204575739 shutdown_connections 2024-04-26T21:08:56.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.396+0000 7f8e02600640 1 -- 172.21.15.26:0/4204575739 wait complete. 2024-04-26T21:08:56.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.398+0000 7f8e02600640 1 Processor -- start 2024-04-26T21:08:56.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.398+0000 7f8e02600640 1 -- start start 2024-04-26T21:08:56.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.399+0000 7f8e02600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8dfc0b3370 0x7f8dfc1f7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:56.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.399+0000 7f8e02600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8dfc160930 0x7f8dfc15a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:56.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.399+0000 7f8e02600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dfc162740 0x7f8dfc0c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:56.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.399+0000 7f8e02600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc0d0920 con 0x7f8dfc160930 2024-04-26T21:08:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.399+0000 7f8e02600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc0ce2d0 con 0x7f8dfc162740 2024-04-26T21:08:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.399+0000 7f8e02600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8dfc0d0aa0 con 0x7f8dfc0b3370 2024-04-26T21:08:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.399+0000 7f8df3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8dfc160930 0x7f8dfc15a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.399+0000 7f8df3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8dfc160930 0x7f8dfc15a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34982/0 (socket says 172.21.15.26:34982) 2024-04-26T21:08:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.399+0000 7f8df3e00640 1 -- 172.21.15.26:0/3077309815 learned_addr learned my addr 172.21.15.26:0/3077309815 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:08:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.400+0000 7f8e01c00640 1 --2- 172.21.15.26:0/3077309815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dfc162740 0x7f8dfc0c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.400+0000 7f8e01200640 1 --2- 172.21.15.26:0/3077309815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8dfc0b3370 0x7f8dfc1f7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.400+0000 7f8df3e00640 1 -- 172.21.15.26:0/3077309815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8dfc0b3370 msgr2=0x7f8dfc1f7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.400+0000 7f8df3e00640 1 --2- 172.21.15.26:0/3077309815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8dfc0b3370 0x7f8dfc1f7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.400+0000 7f8df3e00640 1 -- 172.21.15.26:0/3077309815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dfc162740 msgr2=0x7f8dfc0c5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.400+0000 7f8df3e00640 1 --2- 172.21.15.26:0/3077309815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dfc162740 0x7f8dfc0c5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.400+0000 7f8df3e00640 1 -- 172.21.15.26:0/3077309815 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dfc15acf0 con 0x7f8dfc160930 2024-04-26T21:08:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.400+0000 7f8e01200640 1 --2- 172.21.15.26:0/3077309815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8dfc0b3370 0x7f8dfc1f7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:08:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.400+0000 7f8df3e00640 1 --2- 172.21.15.26:0/3077309815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8dfc160930 0x7f8dfc15a210 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8de4000c00 tx=0x7f8de40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:56.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.400+0000 7f8df9600640 1 -- 172.21.15.26:0/3077309815 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4041020 con 0x7f8dfc160930 2024-04-26T21:08:56.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.400+0000 7f8df9600640 1 -- 172.21.15.26:0/3077309815 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de4030970 con 0x7f8dfc160930 2024-04-26T21:08:56.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.400+0000 7f8df9600640 1 -- 172.21.15.26:0/3077309815 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4037040 con 0x7f8dfc160930 2024-04-26T21:08:56.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.400+0000 7f8e02600640 1 -- 172.21.15.26:0/3077309815 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dfc15afd0 con 0x7f8dfc160930 2024-04-26T21:08:56.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.401+0000 7f8e02600640 1 -- 172.21.15.26:0/3077309815 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dfc15b590 con 0x7f8dfc160930 2024-04-26T21:08:56.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.402+0000 7f8e02600640 1 -- 172.21.15.26:0/3077309815 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dfc06a700 con 0x7f8dfc160930 2024-04-26T21:08:56.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.403+0000 7f8df9600640 1 -- 172.21.15.26:0/3077309815 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8de4044020 con 0x7f8dfc160930 2024-04-26T21:08:56.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.404+0000 7f8df9600640 1 --2- 172.21.15.26:0/3077309815 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8dd806c8f0 0x7f8dd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:08:56.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.404+0000 7f8df9600640 1 -- 172.21.15.26:0/3077309815 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8de40ac660 con 0x7f8dfc160930 2024-04-26T21:08:56.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.404+0000 7f8e01200640 1 --2- 172.21.15.26:0/3077309815 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8dd806c8f0 0x7f8dd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:08:56.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.405+0000 7f8e01200640 1 --2- 172.21.15.26:0/3077309815 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8dd806c8f0 0x7f8dd806edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8dec00a830 tx=0x7f8dec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:08:56.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.408+0000 7f8df9600640 1 -- 172.21.15.26:0/3077309815 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8de404b030 con 0x7f8dfc160930 2024-04-26T21:08:56.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.651+0000 7f8e02600640 1 -- 172.21.15.26:0/3077309815 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8dfc0b5ac0 con 0x7f8dfc160930 2024-04-26T21:08:56.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.651+0000 7f8df9600640 1 -- 172.21.15.26:0/3077309815 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f8de407a9f0 con 0x7f8dfc160930 2024-04-26T21:08:56.653 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:08:56.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.654+0000 7f8e02600640 1 -- 172.21.15.26:0/3077309815 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8dd806c8f0 msgr2=0x7f8dd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:56.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.654+0000 7f8e02600640 1 --2- 172.21.15.26:0/3077309815 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8dd806c8f0 0x7f8dd806edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8dec00a830 tx=0x7f8dec008040 comp rx=0 tx=0).stop 2024-04-26T21:08:56.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.654+0000 7f8e02600640 1 -- 172.21.15.26:0/3077309815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8dfc160930 msgr2=0x7f8dfc15a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:08:56.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.654+0000 7f8e02600640 1 --2- 172.21.15.26:0/3077309815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8dfc160930 0x7f8dfc15a210 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8de4000c00 tx=0x7f8de40046c0 comp rx=0 tx=0).stop 2024-04-26T21:08:56.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.654+0000 7f8e02600640 1 -- 172.21.15.26:0/3077309815 shutdown_connections 2024-04-26T21:08:56.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.654+0000 7f8e02600640 1 --2- 172.21.15.26:0/3077309815 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8dd806c8f0 0x7f8dd806edb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:56.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.654+0000 7f8e02600640 1 --2- 172.21.15.26:0/3077309815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dfc162740 0x7f8dfc0c5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:56.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.654+0000 7f8e02600640 1 --2- 172.21.15.26:0/3077309815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8dfc160930 0x7f8dfc15a210 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:56.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.654+0000 7f8e02600640 1 --2- 172.21.15.26:0/3077309815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8dfc0b3370 0x7f8dfc1f7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:08:56.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.654+0000 7f8e02600640 1 -- 172.21.15.26:0/3077309815 >> 172.21.15.26:0/3077309815 conn(0x7f8dfc06c660 msgr2=0x7f8dfc0ceb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:08:56.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.654+0000 7f8e02600640 1 -- 172.21.15.26:0/3077309815 shutdown_connections 2024-04-26T21:08:56.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56.654+0000 7f8e02600640 1 -- 172.21.15.26:0/3077309815 wait complete. 2024-04-26T21:08:56.674 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:08:56.675 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:08:56.675 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-04-26T21:08:56.675 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:08:56.675 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:08:56.675 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:08:56.675 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:08:56.675 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:08:56.675 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:08:56.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:08:56.675 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-04-26T21:08:56.730 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:08:56.730 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:08:56.730 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:08:56.730 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:08:56.730 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-04-26T21:08:56.730 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:08:56.730 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:08:56.730 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-04-26T21:08:56.730 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-04-26T21:08:56.731 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:08:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-04-26T21:08:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-04-26T21:08:56.731 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:08:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:08:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-04-26T21:08:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T21:08:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42648sec preferred_lft 42648sec 2024-04-26T21:08:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-04-26T21:08:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:08:56.731 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:08:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:08:56.731 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-04-26T21:08:56.732 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T21:08:56.732 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:08:56.732 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-26T21:08:56.732 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add name ceph-brx type bridge 2024-04-26T21:08:56.732 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr flush dev ceph-brx 2024-04-26T21:08:56.732 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set ceph-brx up 2024-04-26T21:08:56.732 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T21:08:56.732 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-26T21:08:56.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:08:57.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:08:57.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:08:57.230 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:08:57.303 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-04-26T21:08:57.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:08:57.306 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-04-26T21:08:57.359 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-04-26T21:08:57.359 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:08:57.359 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:08:57.359 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:08:57.359 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:08:57.360 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:08:57.360 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-26T21:08:57.360 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:08:57.360 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:08:57.360 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:08:57.360 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-26T21:08:57.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:08:58.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:08:58.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:08:58.151 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-04-26T21:08:58.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:08:58.210 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list-id 2024-04-26T21:08:58.266 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:08:58.266 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-26T21:08:58.266 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:08:58.266 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T21:08:58.266 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-26T21:08:58.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:08:58.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:08:58.472 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T21:08:58.473 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:08:58.473 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-26T21:08:58.473 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T21:08:58.473 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T21:08:58.473 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T21:08:58.473 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T21:08:58.473 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T21:08:58.473 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-26T21:08:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:08:58.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:08:58.772 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:08:58.772 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-26T21:08:58.772 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-04-26T21:08:58.772 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T21:08:58.772 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-26T21:08:58.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:08:58.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:08:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:08:58.985 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:08:58.985 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:08:59.029 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:08:59.029 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:08:59.029 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:08:59.125 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:08:59.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:08:59.125 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:08:59.262 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:08:59.262 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:08:59.923 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:08:59.923 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:08:59.923 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-26T21:08:59.923 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with new device syntax: 0@eaf60df9-83a0-4283-ab2c-83d2c02a2067.cephfs=/ 2024-04-26T21:08:59.923 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.26:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-26T21:08:59.929 INFO:teuthology.orchestra.run.smithi026.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:08:59.930 INFO:teuthology.orchestra.run.smithi026.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:08:59.930 INFO:teuthology.orchestra.run.smithi026.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:08:59.930 INFO:teuthology.orchestra.run.smithi026.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:08:59.930 INFO:teuthology.orchestra.run.smithi026.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:08:59.932 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:08:59.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:08:59.932 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:09:00.019 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:09:00.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:09:00.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:00.172 INFO:teuthology.orchestra.run.smithi026.stdout:client4418 2024-04-26T21:09:00.173 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:09:00.173 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:09:00.230 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:09:00.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:09:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:00.382 INFO:teuthology.orchestra.run.smithi026.stdout:client4418 2024-04-26T21:09:00.383 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:09:00.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:09:00.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:00.512 INFO:teuthology.orchestra.run.smithi026.stdout:client4418 2024-04-26T21:09:00.513 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eaf60df9-83a0-4283-ab2c-83d2c02a2067.client4418/status') 2024-04-26T21:09:00.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:09:00.653 INFO:teuthology.orchestra.run.smithi026.stdout:instance: client.4418 (3)172.21.15.26:0/3676951491 2024-04-26T21:09:00.653 INFO:teuthology.orchestra.run.smithi026.stdout:blocklisted: false 2024-04-26T21:09:00.654 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records in 2024-04-26T21:09:00.654 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records out 2024-04-26T21:09:00.654 INFO:teuthology.orchestra.run.smithi026.stderr:70 bytes copied, 0.000108345 s, 646 kB/s 2024-04-26T21:09:00.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:00.666 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-04-26T21:09:00.666 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:09:00.666 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T21:09:00.667 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:09:00.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.876+0000 7ff35f200640 1 Processor -- start 2024-04-26T21:09:00.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.877+0000 7ff35f200640 1 -- start start 2024-04-26T21:09:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.878+0000 7ff35f200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff360162740 0x7ff36006fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.878+0000 7ff35f200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff360162b20 con 0x7ff360160930 2024-04-26T21:09:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.878+0000 7ff35f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3600cd750 con 0x7ff3600b3540 2024-04-26T21:09:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.878+0000 7ff35f200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff3600cda30 con 0x7ff360162740 2024-04-26T21:09:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.879+0000 7ff35d400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff360160930 0x7ff3600b2e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43364/0 (socket says 172.21.15.26:43364) 2024-04-26T21:09:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.879+0000 7ff35d400640 1 -- 172.21.15.26:0/646966599 learned_addr learned my addr 172.21.15.26:0/646966599 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:00.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.879+0000 7ff35ca00640 1 -- 172.21.15.26:0/646966599 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4160948680 0 0) 0x7ff360162b20 con 0x7ff360160930 2024-04-26T21:09:00.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.879+0000 7ff35de00640 1 --2- 172.21.15.26:0/646966599 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff360162740 0x7ff36006fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:00.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.879+0000 7ff35ca00640 1 -- 172.21.15.26:0/646966599 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff344003520 con 0x7ff360160930 2024-04-26T21:09:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.880+0000 7ff35ca00640 1 -- 172.21.15.26:0/646966599 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 993961295 0 0) 0x7ff344003520 con 0x7ff360160930 2024-04-26T21:09:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.880+0000 7ff35ca00640 1 -- 172.21.15.26:0/646966599 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff360162740 msgr2=0x7ff36006fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.880+0000 7ff35ca00640 1 --2- 172.21.15.26:0/646966599 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff360162740 0x7ff36006fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.880+0000 7ff35ca00640 1 -- 172.21.15.26:0/646966599 >> v1:172.21.15.86:6789/0 conn(0x7ff3600b3540 legacy=0x7ff36015f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.880+0000 7ff35ca00640 1 -- 172.21.15.26:0/646966599 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3600cdd10 con 0x7ff360160930 2024-04-26T21:09:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.880+0000 7ff35ca00640 1 -- 172.21.15.26:0/646966599 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7ff34c0019f0 con 0x7ff360160930 2024-04-26T21:09:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.880+0000 7ff35ca00640 1 -- 172.21.15.26:0/646966599 >> v1:172.21.15.26:6789/0 conn(0x7ff360160930 legacy=0x7ff3600b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.880+0000 7ff35ca00640 1 --2- 172.21.15.26:0/646966599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3440039e0 0x7ff344003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.880+0000 7ff35ca00640 1 --2- 172.21.15.26:0/646966599 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff360162740 0x7ff344004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.880+0000 7ff35ca00640 1 --2- 172.21.15.26:0/646966599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff344004d40 0x7ff3440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.880+0000 7ff35ca00640 1 -- 172.21.15.26:0/646966599 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff3600cda30 con 0x7ff3440039e0 2024-04-26T21:09:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.880+0000 7ff35ca00640 1 -- 172.21.15.26:0/646966599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff34400baf0 con 0x7ff344004d40 2024-04-26T21:09:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.880+0000 7ff35ca00640 1 -- 172.21.15.26:0/646966599 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff34400bcf0 con 0x7ff360162740 2024-04-26T21:09:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.880+0000 7ff35de00640 1 --2- 172.21.15.26:0/646966599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3440039e0 0x7ff344003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.881+0000 7ff35d400640 1 --2- 172.21.15.26:0/646966599 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff360162740 0x7ff344004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.881+0000 7ff35e800640 1 --2- 172.21.15.26:0/646966599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff344004d40 0x7ff3440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.881+0000 7ff35de00640 1 -- 172.21.15.26:0/646966599 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff360162740 msgr2=0x7ff344004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.881+0000 7ff35de00640 1 --2- 172.21.15.26:0/646966599 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff360162740 0x7ff344004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.881+0000 7ff35de00640 1 -- 172.21.15.26:0/646966599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff344004d40 msgr2=0x7ff3440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.881+0000 7ff35de00640 1 --2- 172.21.15.26:0/646966599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff344004d40 0x7ff3440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.881+0000 7ff35de00640 1 -- 172.21.15.26:0/646966599 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff344003520 con 0x7ff3440039e0 2024-04-26T21:09:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.881+0000 7ff35e800640 1 --2- 172.21.15.26:0/646966599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff344004d40 0x7ff3440051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:00.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.882+0000 7ff35de00640 1 --2- 172.21.15.26:0/646966599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3440039e0 0x7ff344003e00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff350009ac0 tx=0x7ff344005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc5999eb8bb2527 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:00.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.882+0000 7ff35ca00640 1 -- 172.21.15.26:0/646966599 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff350036070 con 0x7ff3440039e0 2024-04-26T21:09:00.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.882+0000 7ff35ca00640 1 -- 172.21.15.26:0/646966599 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff350031400 con 0x7ff3440039e0 2024-04-26T21:09:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.882+0000 7ff35ca00640 1 -- 172.21.15.26:0/646966599 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff350009690 con 0x7ff3440039e0 2024-04-26T21:09:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.883+0000 7ff35f200640 1 -- 172.21.15.26:0/646966599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3440039e0 msgr2=0x7ff344003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.883+0000 7ff35f200640 1 --2- 172.21.15.26:0/646966599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3440039e0 0x7ff344003e00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff350009ac0 tx=0x7ff344005830 comp rx=0 tx=0).stop 2024-04-26T21:09:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.883+0000 7ff35e800640 1 -- 172.21.15.26:0/646966599 reap_dead start 2024-04-26T21:09:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.883+0000 7ff35f200640 1 -- 172.21.15.26:0/646966599 shutdown_connections 2024-04-26T21:09:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.883+0000 7ff35f200640 1 -- 172.21.15.26:0/646966599 >> 172.21.15.26:0/646966599 conn(0x7ff36006c660 msgr2=0x7ff36006ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.883+0000 7ff35f200640 1 -- 172.21.15.26:0/646966599 shutdown_connections 2024-04-26T21:09:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.883+0000 7ff35f200640 1 -- 172.21.15.26:0/646966599 wait complete. 2024-04-26T21:09:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.885+0000 7ff35f200640 1 Processor -- start 2024-04-26T21:09:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.885+0000 7ff35f200640 1 -- start start 2024-04-26T21:09:00.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.886+0000 7ff35f200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3440039e0 0x7ff3601e8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:00.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.886+0000 7ff35f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3601e7830 0x7ff360073770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:00.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.886+0000 7ff35f200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3601470f0 0x7ff360146430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:00.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.886+0000 7ff35f200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff3600d0920 con 0x7ff3601470f0 2024-04-26T21:09:00.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.886+0000 7ff35f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3600ce2d0 con 0x7ff3601e7830 2024-04-26T21:09:00.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.886+0000 7ff35f200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff3600d0aa0 con 0x7ff3440039e0 2024-04-26T21:09:00.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.886+0000 7ff35e800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3601470f0 0x7ff360146430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:00.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.886+0000 7ff35e800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3601470f0 0x7ff360146430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57958/0 (socket says 172.21.15.26:57958) 2024-04-26T21:09:00.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.886+0000 7ff35e800640 1 -- 172.21.15.26:0/4098140445 learned_addr learned my addr 172.21.15.26:0/4098140445 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:00.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.886+0000 7ff35d400640 1 --2- 172.21.15.26:0/4098140445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3601e7830 0x7ff360073770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:00.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.886+0000 7ff35de00640 1 --2- 172.21.15.26:0/4098140445 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3440039e0 0x7ff3601e8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:00.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.887+0000 7ff35e800640 1 -- 172.21.15.26:0/4098140445 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3440039e0 msgr2=0x7ff3601e8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:00.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.887+0000 7ff35e800640 1 --2- 172.21.15.26:0/4098140445 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3440039e0 0x7ff3601e8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:00.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.887+0000 7ff35e800640 1 -- 172.21.15.26:0/4098140445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3601e7830 msgr2=0x7ff360073770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:00.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.887+0000 7ff35e800640 1 --2- 172.21.15.26:0/4098140445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3601e7830 0x7ff360073770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:00.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.887+0000 7ff35e800640 1 -- 172.21.15.26:0/4098140445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3601e7df0 con 0x7ff3601470f0 2024-04-26T21:09:00.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.887+0000 7ff35e800640 1 --2- 172.21.15.26:0/4098140445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3601470f0 0x7ff360146430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff36006e720 tx=0x7ff35800d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:00.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.887+0000 7ff356000640 1 -- 172.21.15.26:0/4098140445 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff358005480 con 0x7ff3601470f0 2024-04-26T21:09:00.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.887+0000 7ff356000640 1 -- 172.21.15.26:0/4098140445 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff358007030 con 0x7ff3601470f0 2024-04-26T21:09:00.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.887+0000 7ff35f200640 1 -- 172.21.15.26:0/4098140445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff36015df90 con 0x7ff3601470f0 2024-04-26T21:09:00.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.888+0000 7ff356000640 1 -- 172.21.15.26:0/4098140445 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff358007440 con 0x7ff3601470f0 2024-04-26T21:09:00.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.888+0000 7ff35f200640 1 -- 172.21.15.26:0/4098140445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3601e8750 con 0x7ff3601470f0 2024-04-26T21:09:00.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.889+0000 7ff35f200640 1 -- 172.21.15.26:0/4098140445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff36015dd80 con 0x7ff3601470f0 2024-04-26T21:09:00.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.890+0000 7ff356000640 1 -- 172.21.15.26:0/4098140445 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3580067d0 con 0x7ff3601470f0 2024-04-26T21:09:00.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.891+0000 7ff356000640 1 --2- 172.21.15.26:0/4098140445 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff34006c920 0x7ff34006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:00.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.891+0000 7ff356000640 1 -- 172.21.15.26:0/4098140445 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff35808b520 con 0x7ff3601470f0 2024-04-26T21:09:00.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.891+0000 7ff35de00640 1 --2- 172.21.15.26:0/4098140445 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff34006c920 0x7ff34006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:00.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.892+0000 7ff35de00640 1 --2- 172.21.15.26:0/4098140445 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff34006c920 0x7ff34006ede0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff350033000 tx=0x7ff350008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:00.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:00.895+0000 7ff356000640 1 -- 172.21.15.26:0/4098140445 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff3580598b0 con 0x7ff3601470f0 2024-04-26T21:09:01.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:01.141+0000 7ff35f200640 1 -- 172.21.15.26:0/4098140445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff36006a700 con 0x7ff3601470f0 2024-04-26T21:09:01.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:01.142+0000 7ff356000640 1 -- 172.21.15.26:0/4098140445 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7ff3580591b0 con 0x7ff3601470f0 2024-04-26T21:09:01.143 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:09:01.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:01.144+0000 7ff35f200640 1 -- 172.21.15.26:0/4098140445 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff34006c920 msgr2=0x7ff34006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:01.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:01.144+0000 7ff35f200640 1 --2- 172.21.15.26:0/4098140445 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff34006c920 0x7ff34006ede0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff350033000 tx=0x7ff350008030 comp rx=0 tx=0).stop 2024-04-26T21:09:01.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:01.144+0000 7ff35f200640 1 -- 172.21.15.26:0/4098140445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3601470f0 msgr2=0x7ff360146430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:01.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:01.144+0000 7ff35f200640 1 --2- 172.21.15.26:0/4098140445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3601470f0 0x7ff360146430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff36006e720 tx=0x7ff35800d890 comp rx=0 tx=0).stop 2024-04-26T21:09:01.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:01.144+0000 7ff35f200640 1 -- 172.21.15.26:0/4098140445 shutdown_connections 2024-04-26T21:09:01.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:01.144+0000 7ff35f200640 1 --2- 172.21.15.26:0/4098140445 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff34006c920 0x7ff34006ede0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:01.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:01.144+0000 7ff35f200640 1 --2- 172.21.15.26:0/4098140445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3601470f0 0x7ff360146430 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:01.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:01.144+0000 7ff35f200640 1 --2- 172.21.15.26:0/4098140445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3601e7830 0x7ff360073770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:01.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:01.144+0000 7ff35f200640 1 --2- 172.21.15.26:0/4098140445 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3440039e0 0x7ff3601e8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:01.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:01.144+0000 7ff35f200640 1 -- 172.21.15.26:0/4098140445 >> 172.21.15.26:0/4098140445 conn(0x7ff36006c660 msgr2=0x7ff3600b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:01.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:01.145+0000 7ff35f200640 1 -- 172.21.15.26:0/4098140445 shutdown_connections 2024-04-26T21:09:01.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:01.145+0000 7ff35f200640 1 -- 172.21.15.26:0/4098140445 wait complete. 2024-04-26T21:09:01.165 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:09:01.165 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:09:01.165 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-04-26T21:09:01.165 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-26T21:09:01.165 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T21:09:01.165 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:09:01.165 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:09:01.166 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:09:01.166 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-26T21:09:01.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:01.166 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-04-26T21:09:01.221 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:09:01.221 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:09:01.221 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:09:01.222 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:01.222 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-04-26T21:09:01.222 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:01.222 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:09:01.222 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-04-26T21:09:01.222 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-04-26T21:09:01.222 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:09:01.222 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-04-26T21:09:01.222 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-04-26T21:09:01.222 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:09:01.222 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:09:01.222 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-04-26T21:09:01.222 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T21:09:01.223 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42643sec preferred_lft 42643sec 2024-04-26T21:09:01.223 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-04-26T21:09:01.223 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:01.223 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:09:01.223 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:09:01.223 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-04-26T21:09:01.223 INFO:teuthology.orchestra.run.smithi026.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:09:01.223 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 6a:d4:79:b9:a0:21 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:09:01.223 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:09:01.223 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:01.223 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::7072:11ff:fee4:1709/64 scope link 2024-04-26T21:09:01.223 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:01.223 INFO:teuthology.orchestra.run.smithi026.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:09:01.223 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 6a:d4:79:b9:a0:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:09:01.223 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::68d4:79ff:feb9:a021/64 scope link 2024-04-26T21:09:01.223 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:01.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:01.224 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:09:01.310 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-04-26T21:09:01.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:01.314 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-04-26T21:09:01.372 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-04-26T21:09:01.372 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:09:01.372 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:09:01.372 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:09:01.372 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:09:01.373 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:09:01.373 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-26T21:09:01.373 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:09:01.373 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:09:01.373 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:09:01.373 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-26T21:09:01.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:09:01.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:01.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:01.600 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-04-26T21:09:01.651 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:09:01.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:01.651 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list-id 2024-04-26T21:09:01.707 INFO:teuthology.orchestra.run.smithi026.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-26T21:09:01.709 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:09:01.709 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-26T21:09:01.709 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:09:01.709 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-26T21:09:01.709 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-26T21:09:01.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:09:01.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:01.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:01.906 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T21:09:01.992 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:09:01.992 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:09:01.992 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:09:01.992 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:01.992 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-04-26T21:09:01.992 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:01.993 INFO:teuthology.orchestra.run.smithi026.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:09:01.993 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 76:ea:30:96:32:0f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T21:09:01.993 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T21:09:01.993 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:01.993 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::74ea:30ff:fe96:320f/64 scope link 2024-04-26T21:09:01.993 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:01.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:01.997 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T21:09:02.087 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:09:02.087 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:09:02.087 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:09:02.087 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:02.087 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-04-26T21:09:02.087 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:02.087 INFO:teuthology.orchestra.run.smithi026.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:09:02.087 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 76:ea:30:96:32:0f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T21:09:02.087 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T21:09:02.087 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:02.088 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::74ea:30ff:fe96:320f/64 scope link 2024-04-26T21:09:02.088 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:02.091 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-26T21:09:02.091 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:09:02.092 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-26T21:09:02.092 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-26T21:09:02.092 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T21:09:02.092 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-26T21:09:02.092 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-26T21:09:02.092 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-26T21:09:02.092 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-26T21:09:02.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:09:02.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:02.382 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:09:02.382 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-26T21:09:02.383 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.1 up 2024-04-26T21:09:02.383 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T21:09:02.383 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-26T21:09:02.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:09:02.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:02.606 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:09:02.606 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-26T21:09:02.654 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-26T21:09:02.654 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:09:02.654 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-26T21:09:02.750 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-04-26T21:09:02.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:02.750 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:09:02.888 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:09:02.888 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:09:02.924 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:09:02.924 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:09:02.924 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-26T21:09:02.924 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with new device syntax: 2@eaf60df9-83a0-4283-ab2c-83d2c02a2067.cephfs=/ 2024-04-26T21:09:02.924 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.26:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-26T21:09:02.926 INFO:teuthology.orchestra.run.smithi026.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-26T21:09:02.926 INFO:teuthology.orchestra.run.smithi026.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:09:02.926 INFO:teuthology.orchestra.run.smithi026.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:09:02.926 INFO:teuthology.orchestra.run.smithi026.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:09:02.926 INFO:teuthology.orchestra.run.smithi026.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:09:02.930 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:09:02.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:02.931 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-26T21:09:03.014 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:09:03.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:09:03.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:03.138 INFO:teuthology.orchestra.run.smithi026.stdout:client4668 2024-04-26T21:09:03.138 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:09:03.138 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:09:03.192 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:09:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:09:03.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:03.317 INFO:teuthology.orchestra.run.smithi026.stdout:client4668 2024-04-26T21:09:03.317 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:09:03.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:09:03.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:03.444 INFO:teuthology.orchestra.run.smithi026.stdout:client4668 2024-04-26T21:09:03.445 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eaf60df9-83a0-4283-ab2c-83d2c02a2067.client4668/status') 2024-04-26T21:09:03.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:09:03.586 INFO:teuthology.orchestra.run.smithi026.stdout:instance: client.4668 (3)192.168.0.2:0/1490283406 2024-04-26T21:09:03.586 INFO:teuthology.orchestra.run.smithi026.stdout:blocklisted: false 2024-04-26T21:09:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records in 2024-04-26T21:09:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records out 2024-04-26T21:09:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:69 bytes copied, 0.000114018 s, 605 kB/s 2024-04-26T21:09:03.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:03.603 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-04-26T21:09:03.603 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:09:03.603 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T21:09:03.603 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:09:03.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.815+0000 7fa7d2e00640 1 Processor -- start 2024-04-26T21:09:03.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.816+0000 7fa7d2e00640 1 -- start start 2024-04-26T21:09:03.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.817+0000 7fa7d2e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa7cc146990 0x7fa7cc0b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:03.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.817+0000 7fa7d2e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa7cc06ec20 0x7fa7cc0d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:03.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.817+0000 7fa7d2e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7cc0b4480 con 0x7fa7cc0d1210 2024-04-26T21:09:03.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.817+0000 7fa7d2e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fa7cc146d70 con 0x7fa7cc146990 2024-04-26T21:09:03.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.817+0000 7fa7d2e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa7cc0d1640 con 0x7fa7cc06ec20 2024-04-26T21:09:03.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.818+0000 7fa7cbe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa7cc146990 0x7fa7cc0b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:03.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.818+0000 7fa7cbe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa7cc146990 0x7fa7cc0b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57972/0 (socket says 172.21.15.26:57972) 2024-04-26T21:09:03.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.818+0000 7fa7cbe00640 1 -- 172.21.15.26:0/3883913535 learned_addr learned my addr 172.21.15.26:0/3883913535 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:03.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.818+0000 7fa7cb400640 1 --2- 172.21.15.26:0/3883913535 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa7cc06ec20 0x7fa7cc0d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:03.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.818+0000 7fa7cbe00640 1 -- 172.21.15.26:0/3883913535 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa7cc06ec20 msgr2=0x7fa7cc0d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:03.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.818+0000 7fa7cbe00640 1 --2- 172.21.15.26:0/3883913535 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa7cc06ec20 0x7fa7cc0d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:03.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.818+0000 7fa7cbe00640 1 -- 172.21.15.26:0/3883913535 >> v1:172.21.15.86:6789/0 conn(0x7fa7cc0d1210 legacy=0x7fa7cc072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:03.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.818+0000 7fa7cbe00640 1 -- 172.21.15.26:0/3883913535 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7cc073b70 con 0x7fa7cc146990 2024-04-26T21:09:03.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.819+0000 7fa7cbe00640 1 --2- 172.21.15.26:0/3883913535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7cc146990 0x7fa7cc0b3f40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009e50 tx=0x7fa7bc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d905ed72c43beb58 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:03.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.820+0000 7fa7caa00640 1 -- 172.21.15.26:0/3883913535 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc036070 con 0x7fa7cc146990 2024-04-26T21:09:03.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.820+0000 7fa7caa00640 1 -- 172.21.15.26:0/3883913535 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7bc031600 con 0x7fa7cc146990 2024-04-26T21:09:03.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.820+0000 7fa7caa00640 1 -- 172.21.15.26:0/3883913535 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc0319d0 con 0x7fa7cc146990 2024-04-26T21:09:03.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.821+0000 7fa7d2e00640 1 -- 172.21.15.26:0/3883913535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7cc146990 msgr2=0x7fa7cc0b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:03.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.821+0000 7fa7d2e00640 1 --2- 172.21.15.26:0/3883913535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7cc146990 0x7fa7cc0b3f40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009e50 tx=0x7fa7bc02a710 comp rx=0 tx=0).stop 2024-04-26T21:09:03.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.821+0000 7fa7d2e00640 1 -- 172.21.15.26:0/3883913535 shutdown_connections 2024-04-26T21:09:03.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.821+0000 7fa7d2e00640 1 --2- 172.21.15.26:0/3883913535 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa7cc06ec20 0x7fa7cc0d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:03.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.821+0000 7fa7d2e00640 1 --2- 172.21.15.26:0/3883913535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7cc146990 0x7fa7cc0b3f40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:03.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.821+0000 7fa7d2e00640 1 -- 172.21.15.26:0/3883913535 >> 172.21.15.26:0/3883913535 conn(0x7fa7cc0cd840 msgr2=0x7fa7cc0cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:03.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.821+0000 7fa7d2e00640 1 -- 172.21.15.26:0/3883913535 shutdown_connections 2024-04-26T21:09:03.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.821+0000 7fa7d2e00640 1 -- 172.21.15.26:0/3883913535 wait complete. 2024-04-26T21:09:03.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.823+0000 7fa7d2e00640 1 Processor -- start 2024-04-26T21:09:03.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.823+0000 7fa7d2e00640 1 -- start start 2024-04-26T21:09:03.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.824+0000 7fa7d2e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa7cc06ec20 0x7fa7cc1452f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:03.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.824+0000 7fa7d2e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7cc0d1210 0x7fa7cc1e83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:03.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.824+0000 7fa7d2e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7cc146990 0x7fa7cc1e8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:03.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.824+0000 7fa7d2e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc06e420 con 0x7fa7cc146990 2024-04-26T21:09:03.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.824+0000 7fa7d2e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc074130 con 0x7fa7cc0d1210 2024-04-26T21:09:03.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.824+0000 7fa7d2e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa7cc06e700 con 0x7fa7cc06ec20 2024-04-26T21:09:03.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.824+0000 7fa7d0e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7cc146990 0x7fa7cc1e8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:03.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.824+0000 7fa7d0e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7cc146990 0x7fa7cc1e8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57978/0 (socket says 172.21.15.26:57978) 2024-04-26T21:09:03.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.824+0000 7fa7d0e00640 1 -- 172.21.15.26:0/1411424248 learned_addr learned my addr 172.21.15.26:0/1411424248 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:03.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.825+0000 7fa7cbe00640 1 --2- 172.21.15.26:0/1411424248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa7cc06ec20 0x7fa7cc1452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:03.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.825+0000 7fa7cb400640 1 --2- 172.21.15.26:0/1411424248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7cc0d1210 0x7fa7cc1e83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:03.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.825+0000 7fa7d0e00640 1 -- 172.21.15.26:0/1411424248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa7cc06ec20 msgr2=0x7fa7cc1452f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:03.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.825+0000 7fa7d0e00640 1 --2- 172.21.15.26:0/1411424248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa7cc06ec20 0x7fa7cc1452f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:03.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.825+0000 7fa7d0e00640 1 -- 172.21.15.26:0/1411424248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7cc0d1210 msgr2=0x7fa7cc1e83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:03.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.825+0000 7fa7d0e00640 1 --2- 172.21.15.26:0/1411424248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7cc0d1210 0x7fa7cc1e83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:03.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.825+0000 7fa7d0e00640 1 -- 172.21.15.26:0/1411424248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7cc0ac420 con 0x7fa7cc146990 2024-04-26T21:09:03.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.825+0000 7fa7cb400640 1 --2- 172.21.15.26:0/1411424248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7cc0d1210 0x7fa7cc1e83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:03.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.825+0000 7fa7d0e00640 1 --2- 172.21.15.26:0/1411424248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7cc146990 0x7fa7cc1e8900 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400c390 tx=0x7fa7c400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:03.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.825+0000 7fa7c3e00640 1 -- 172.21.15.26:0/1411424248 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4017070 con 0x7fa7cc146990 2024-04-26T21:09:03.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.825+0000 7fa7c3e00640 1 -- 172.21.15.26:0/1411424248 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7c40052d0 con 0x7fa7cc146990 2024-04-26T21:09:03.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.825+0000 7fa7c3e00640 1 -- 172.21.15.26:0/1411424248 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4003810 con 0x7fa7cc146990 2024-04-26T21:09:03.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.825+0000 7fa7d2e00640 1 -- 172.21.15.26:0/1411424248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7cc0ac700 con 0x7fa7cc146990 2024-04-26T21:09:03.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.826+0000 7fa7d2e00640 1 -- 172.21.15.26:0/1411424248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7cc0accc0 con 0x7fa7cc146990 2024-04-26T21:09:03.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.827+0000 7fa7d2e00640 1 -- 172.21.15.26:0/1411424248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7cc06a700 con 0x7fa7cc146990 2024-04-26T21:09:03.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.828+0000 7fa7c3e00640 1 -- 172.21.15.26:0/1411424248 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7c401a070 con 0x7fa7cc146990 2024-04-26T21:09:03.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.829+0000 7fa7c3e00640 1 --2- 172.21.15.26:0/1411424248 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa7a806c8f0 0x7fa7a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:03.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.829+0000 7fa7c3e00640 1 -- 172.21.15.26:0/1411424248 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa7c408ae80 con 0x7fa7cc146990 2024-04-26T21:09:03.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.829+0000 7fa7cbe00640 1 --2- 172.21.15.26:0/1411424248 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa7a806c8f0 0x7fa7a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:03.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.830+0000 7fa7cbe00640 1 --2- 172.21.15.26:0/1411424248 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa7a806c8f0 0x7fa7a806edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc033000 tx=0x7fa7bc009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:03.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:03.834+0000 7fa7c3e00640 1 -- 172.21.15.26:0/1411424248 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa7c40592a0 con 0x7fa7cc146990 2024-04-26T21:09:04.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:04.086+0000 7fa7d2e00640 1 -- 172.21.15.26:0/1411424248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa7cc06fd50 con 0x7fa7cc146990 2024-04-26T21:09:04.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:04.086+0000 7fa7c3e00640 1 -- 172.21.15.26:0/1411424248 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fa7c4058ba0 con 0x7fa7cc146990 2024-04-26T21:09:04.088 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:09:04.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:04.089+0000 7fa7d2e00640 1 -- 172.21.15.26:0/1411424248 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa7a806c8f0 msgr2=0x7fa7a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:04.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:04.089+0000 7fa7d2e00640 1 --2- 172.21.15.26:0/1411424248 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa7a806c8f0 0x7fa7a806edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc033000 tx=0x7fa7bc009000 comp rx=0 tx=0).stop 2024-04-26T21:09:04.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:04.089+0000 7fa7d2e00640 1 -- 172.21.15.26:0/1411424248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7cc146990 msgr2=0x7fa7cc1e8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:04.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:04.089+0000 7fa7d2e00640 1 --2- 172.21.15.26:0/1411424248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7cc146990 0x7fa7cc1e8900 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400c390 tx=0x7fa7c400c850 comp rx=0 tx=0).stop 2024-04-26T21:09:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:04.090+0000 7fa7d2e00640 1 -- 172.21.15.26:0/1411424248 shutdown_connections 2024-04-26T21:09:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:04.090+0000 7fa7d2e00640 1 --2- 172.21.15.26:0/1411424248 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa7a806c8f0 0x7fa7a806edb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:04.090+0000 7fa7d2e00640 1 --2- 172.21.15.26:0/1411424248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7cc146990 0x7fa7cc1e8900 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:04.090+0000 7fa7d2e00640 1 --2- 172.21.15.26:0/1411424248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7cc0d1210 0x7fa7cc1e83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:04.090+0000 7fa7d2e00640 1 --2- 172.21.15.26:0/1411424248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa7cc06ec20 0x7fa7cc1452f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:04.090+0000 7fa7d2e00640 1 -- 172.21.15.26:0/1411424248 >> 172.21.15.26:0/1411424248 conn(0x7fa7cc0cd840 msgr2=0x7fa7cc0b6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:04.090+0000 7fa7d2e00640 1 -- 172.21.15.26:0/1411424248 shutdown_connections 2024-04-26T21:09:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:04.090+0000 7fa7d2e00640 1 -- 172.21.15.26:0/1411424248 wait complete. 2024-04-26T21:09:04.111 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:09:04.111 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:09:04.111 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-26T21:09:04.111 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T21:09:04.111 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T21:09:04.111 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:09:04.111 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:09:04.111 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:09:04.111 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:09:04.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:04.112 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-26T21:09:04.169 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:09:04.170 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:09:04.170 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:09:04.170 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:04.170 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-26T21:09:04.170 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:04.170 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:09:04.170 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:09:04.170 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-26T21:09:04.170 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:09:04.170 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:09:04.170 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-26T21:09:04.170 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:09:04.171 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:09:04.171 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:09:04.171 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:09:04.171 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T21:09:04.171 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42615sec preferred_lft 42615sec 2024-04-26T21:09:04.171 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-26T21:09:04.171 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:04.172 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T21:09:04.172 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:09:04.172 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-26T21:09:04.172 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add name ceph-brx type bridge 2024-04-26T21:09:04.172 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr flush dev ceph-brx 2024-04-26T21:09:04.172 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set ceph-brx up 2024-04-26T21:09:04.172 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T21:09:04.173 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-26T21:09:04.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:09:04.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:04.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:04.661 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:09:04.731 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-26T21:09:04.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:04.735 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-26T21:09:04.795 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-26T21:09:04.796 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:09:04.796 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:09:04.796 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:09:04.796 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:09:04.796 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:09:04.796 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-26T21:09:04.796 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:09:04.796 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:09:04.796 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:09:04.796 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-26T21:09:04.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:09:05.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:05.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:05.548 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-26T21:09:05.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:05.599 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-04-26T21:09:05.649 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:09:05.649 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-26T21:09:05.649 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:09:05.649 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-26T21:09:05.649 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-26T21:09:05.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:09:05.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:05.829 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-26T21:09:05.829 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:09:05.829 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-26T21:09:05.829 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-26T21:09:05.829 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T21:09:05.830 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-26T21:09:05.830 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-26T21:09:05.830 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-26T21:09:05.830 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-26T21:09:05.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:09:06.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:06.120 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:09:06.120 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-26T21:09:06.120 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-26T21:09:06.120 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T21:09:06.120 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-26T21:09:06.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:09:06.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:06.320 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:09:06.320 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T21:09:06.364 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T21:09:06.364 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:09:06.364 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:09:06.462 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-26T21:09:06.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:06.463 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:09:06.588 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:09:06.589 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:09:06.722 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:09:06.730 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:09:07.370 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:09:07.370 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:09:07.370 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-26T21:09:07.370 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with new device syntax: 1@eaf60df9-83a0-4283-ab2c-83d2c02a2067.cephfs=/ 2024-04-26T21:09:07.371 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.26:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-26T21:09:07.373 INFO:teuthology.orchestra.run.smithi086.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T21:09:07.373 INFO:teuthology.orchestra.run.smithi086.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:09:07.373 INFO:teuthology.orchestra.run.smithi086.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:09:07.373 INFO:teuthology.orchestra.run.smithi086.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:09:07.373 INFO:teuthology.orchestra.run.smithi086.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:09:07.378 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:09:07.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:07.378 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T21:09:07.459 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:09:07.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:09:07.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:07.579 INFO:teuthology.orchestra.run.smithi086.stdout:client4677 2024-04-26T21:09:07.580 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:09:07.580 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:09:07.628 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:09:07.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:09:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:07.792 INFO:teuthology.orchestra.run.smithi086.stdout:client4677 2024-04-26T21:09:07.793 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:09:07.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:09:07.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:07.908 INFO:teuthology.orchestra.run.smithi086.stdout:client4677 2024-04-26T21:09:07.909 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eaf60df9-83a0-4283-ab2c-83d2c02a2067.client4677/status') 2024-04-26T21:09:08.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:09:08.041 INFO:teuthology.orchestra.run.smithi086.stdout:instance: client.4677 (3)172.21.15.86:0/374798717 2024-04-26T21:09:08.041 INFO:teuthology.orchestra.run.smithi086.stdout:blocklisted: false 2024-04-26T21:09:08.042 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records in 2024-04-26T21:09:08.042 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records out 2024-04-26T21:09:08.042 INFO:teuthology.orchestra.run.smithi086.stderr:69 bytes copied, 0.000103853 s, 664 kB/s 2024-04-26T21:09:08.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:08.054 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-04-26T21:09:08.054 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:09:08.054 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T21:09:08.054 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:09:08.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.275+0000 7f5f0da00640 1 Processor -- start 2024-04-26T21:09:08.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.275+0000 7f5f0da00640 1 -- start start 2024-04-26T21:09:08.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.276+0000 7f5f0da00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5f080d13e0 0x7f5f08072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:08.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.276+0000 7f5f0da00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f08073b70 con 0x7f5f0806ec20 2024-04-26T21:09:08.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.276+0000 7f5f0da00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f08073e50 con 0x7f5f08146990 2024-04-26T21:09:08.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.276+0000 7f5f0da00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f5f08146d70 con 0x7f5f080d13e0 2024-04-26T21:09:08.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.277+0000 7f5f07400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5f080d13e0 0x7f5f08072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:08.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.277+0000 7f5f06000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5f0806ec20 0x7f5f080d0cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43372/0 (socket says 172.21.15.26:43372) 2024-04-26T21:09:08.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.277+0000 7f5f06000640 1 -- 172.21.15.26:0/1672060042 learned_addr learned my addr 172.21.15.26:0/1672060042 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:08.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.278+0000 7f5f07400640 1 -- 172.21.15.26:0/1672060042 >> v1:172.21.15.86:6789/0 conn(0x7f5f08146990 legacy=0x7f5f080b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:08.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.278+0000 7f5f07400640 1 -- 172.21.15.26:0/1672060042 >> v1:172.21.15.26:6789/0 conn(0x7f5f0806ec20 legacy=0x7f5f080d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:08.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.278+0000 7f5f07400640 1 -- 172.21.15.26:0/1672060042 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f08074130 con 0x7f5f080d13e0 2024-04-26T21:09:08.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.279+0000 7f5f07400640 1 --2- 172.21.15.26:0/1672060042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f080d13e0 0x7f5f08072100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009d90 tx=0x7f5f0002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe463f997314a6a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:08.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.279+0000 7f5f05600640 1 -- 172.21.15.26:0/1672060042 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f00007740 con 0x7f5f080d13e0 2024-04-26T21:09:08.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.279+0000 7f5f05600640 1 -- 172.21.15.26:0/1672060042 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f000078e0 con 0x7f5f080d13e0 2024-04-26T21:09:08.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.279+0000 7f5f05600640 1 -- 172.21.15.26:0/1672060042 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f00007cb0 con 0x7f5f080d13e0 2024-04-26T21:09:08.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.279+0000 7f5f0da00640 1 -- 172.21.15.26:0/1672060042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f080d13e0 msgr2=0x7f5f08072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:08.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.279+0000 7f5f0da00640 1 --2- 172.21.15.26:0/1672060042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f080d13e0 0x7f5f08072100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009d90 tx=0x7f5f0002a6f0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.280+0000 7f5f0da00640 1 -- 172.21.15.26:0/1672060042 shutdown_connections 2024-04-26T21:09:08.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.280+0000 7f5f0da00640 1 --2- 172.21.15.26:0/1672060042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f080d13e0 0x7f5f08072100 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.280+0000 7f5f0da00640 1 -- 172.21.15.26:0/1672060042 >> 172.21.15.26:0/1672060042 conn(0x7f5f080cd840 msgr2=0x7f5f080cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:08.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.280+0000 7f5f0da00640 1 -- 172.21.15.26:0/1672060042 shutdown_connections 2024-04-26T21:09:08.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.280+0000 7f5f0da00640 1 -- 172.21.15.26:0/1672060042 wait complete. 2024-04-26T21:09:08.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.282+0000 7f5f0da00640 1 Processor -- start 2024-04-26T21:09:08.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.282+0000 7f5f0da00640 1 -- start start 2024-04-26T21:09:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.283+0000 7f5f0da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f0806ec20 0x7f5f081454c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.283+0000 7f5f0da00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f080d13e0 0x7f5f081570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.283+0000 7f5f0da00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f08146990 0x7f5f080cff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.283+0000 7f5f0da00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0806f0b0 con 0x7f5f080d13e0 2024-04-26T21:09:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.283+0000 7f5f0da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0806e700 con 0x7f5f0806ec20 2024-04-26T21:09:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.283+0000 7f5f0da00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5f0806f230 con 0x7f5f08146990 2024-04-26T21:09:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.283+0000 7f5f06000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f080d13e0 0x7f5f081570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.283+0000 7f5f06000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f080d13e0 0x7f5f081570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57990/0 (socket says 172.21.15.26:57990) 2024-04-26T21:09:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.283+0000 7f5f06000640 1 -- 172.21.15.26:0/3217005301 learned_addr learned my addr 172.21.15.26:0/3217005301 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.283+0000 7f5f07400640 1 --2- 172.21.15.26:0/3217005301 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f08146990 0x7f5f080cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.283+0000 7f5f06a00640 1 --2- 172.21.15.26:0/3217005301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f0806ec20 0x7f5f081454c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:08.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.283+0000 7f5f06000640 1 -- 172.21.15.26:0/3217005301 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f08146990 msgr2=0x7f5f080cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:08.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.283+0000 7f5f06000640 1 --2- 172.21.15.26:0/3217005301 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f08146990 0x7f5f080cff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.284+0000 7f5f06000640 1 -- 172.21.15.26:0/3217005301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f0806ec20 msgr2=0x7f5f081454c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:08.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.284+0000 7f5f06000640 1 --2- 172.21.15.26:0/3217005301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f0806ec20 0x7f5f081454c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.284+0000 7f5f06000640 1 -- 172.21.15.26:0/3217005301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0815e6b0 con 0x7f5f080d13e0 2024-04-26T21:09:08.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.284+0000 7f5f06000640 1 --2- 172.21.15.26:0/3217005301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f080d13e0 0x7f5f081570c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400c430 tx=0x7f5ef400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:08.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.284+0000 7f5efea00640 1 -- 172.21.15.26:0/3217005301 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4017070 con 0x7f5f080d13e0 2024-04-26T21:09:08.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.284+0000 7f5efea00640 1 -- 172.21.15.26:0/3217005301 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef4005ce0 con 0x7f5f080d13e0 2024-04-26T21:09:08.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.284+0000 7f5efea00640 1 -- 172.21.15.26:0/3217005301 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4007310 con 0x7f5f080d13e0 2024-04-26T21:09:08.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.284+0000 7f5f0da00640 1 -- 172.21.15.26:0/3217005301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f0815e990 con 0x7f5f080d13e0 2024-04-26T21:09:08.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.284+0000 7f5f0da00640 1 -- 172.21.15.26:0/3217005301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f0815ef50 con 0x7f5f080d13e0 2024-04-26T21:09:08.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.286+0000 7f5f0da00640 1 -- 172.21.15.26:0/3217005301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f0806a700 con 0x7f5f080d13e0 2024-04-26T21:09:08.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.287+0000 7f5efea00640 1 -- 172.21.15.26:0/3217005301 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ef401a070 con 0x7f5f080d13e0 2024-04-26T21:09:08.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.288+0000 7f5efea00640 1 --2- 172.21.15.26:0/3217005301 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5ee806c8f0 0x7f5ee806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:08.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.288+0000 7f5efea00640 1 -- 172.21.15.26:0/3217005301 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5ef408afc0 con 0x7f5f080d13e0 2024-04-26T21:09:08.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.288+0000 7f5f06a00640 1 --2- 172.21.15.26:0/3217005301 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5ee806c8f0 0x7f5ee806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:08.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.289+0000 7f5f06a00640 1 --2- 172.21.15.26:0/3217005301 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5ee806c8f0 0x7f5ee806edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8001640 tx=0x7f5ef8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:08.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.293+0000 7f5efea00640 1 -- 172.21.15.26:0/3217005301 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5ef40592c0 con 0x7f5f080d13e0 2024-04-26T21:09:08.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.546+0000 7f5f0da00640 1 -- 172.21.15.26:0/3217005301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5f0806fd50 con 0x7f5f080d13e0 2024-04-26T21:09:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.547+0000 7f5efea00640 1 -- 172.21.15.26:0/3217005301 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f5ef4058bc0 con 0x7f5f080d13e0 2024-04-26T21:09:08.549 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:09:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.550+0000 7f5f0da00640 1 -- 172.21.15.26:0/3217005301 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5ee806c8f0 msgr2=0x7f5ee806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.550+0000 7f5f0da00640 1 --2- 172.21.15.26:0/3217005301 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5ee806c8f0 0x7f5ee806edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8001640 tx=0x7f5ef8009040 comp rx=0 tx=0).stop 2024-04-26T21:09:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.550+0000 7f5f0da00640 1 -- 172.21.15.26:0/3217005301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f080d13e0 msgr2=0x7f5f081570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.550+0000 7f5f0da00640 1 --2- 172.21.15.26:0/3217005301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f080d13e0 0x7f5f081570c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400c430 tx=0x7f5ef400c8f0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.550+0000 7f5f0da00640 1 -- 172.21.15.26:0/3217005301 shutdown_connections 2024-04-26T21:09:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.550+0000 7f5f0da00640 1 --2- 172.21.15.26:0/3217005301 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5ee806c8f0 0x7f5ee806edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.550+0000 7f5f0da00640 1 --2- 172.21.15.26:0/3217005301 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5f08146990 0x7f5f080cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.550+0000 7f5f0da00640 1 --2- 172.21.15.26:0/3217005301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f080d13e0 0x7f5f081570c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.550+0000 7f5f0da00640 1 --2- 172.21.15.26:0/3217005301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f0806ec20 0x7f5f081454c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.550+0000 7f5f0da00640 1 -- 172.21.15.26:0/3217005301 >> 172.21.15.26:0/3217005301 conn(0x7f5f080cd840 msgr2=0x7f5f080b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:08.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.550+0000 7f5f0da00640 1 -- 172.21.15.26:0/3217005301 shutdown_connections 2024-04-26T21:09:08.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:08.550+0000 7f5f0da00640 1 -- 172.21.15.26:0/3217005301 wait complete. 2024-04-26T21:09:08.572 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:09:08.573 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:09:08.573 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-26T21:09:08.573 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-26T21:09:08.573 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T21:09:08.573 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:09:08.573 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:09:08.573 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:09:08.573 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-26T21:09:08.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:08.573 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-26T21:09:08.631 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:09:08.631 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:09:08.631 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:09:08.631 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:08.631 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-26T21:09:08.632 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:08.632 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:09:08.632 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:09:08.632 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-26T21:09:08.632 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:09:08.632 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:09:08.632 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-26T21:09:08.632 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:09:08.632 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:09:08.632 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:09:08.632 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:09:08.633 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T21:09:08.633 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42611sec preferred_lft 42611sec 2024-04-26T21:09:08.633 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-26T21:09:08.633 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:08.633 INFO:teuthology.orchestra.run.smithi086.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:09:08.633 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 06:aa:74:50:34:bc brd ff:ff:ff:ff:ff:ff 2024-04-26T21:09:08.633 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:09:08.633 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:08.633 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::280f:d8ff:fe48:98ec/64 scope link 2024-04-26T21:09:08.633 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:08.633 INFO:teuthology.orchestra.run.smithi086.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:09:08.633 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 06:aa:74:50:34:bc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:09:08.634 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::4aa:74ff:fe50:34bc/64 scope link 2024-04-26T21:09:08.634 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:08.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:08.635 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:09:08.711 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-26T21:09:08.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:08.715 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-26T21:09:08.766 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-26T21:09:08.767 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:09:08.767 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T21:09:08.767 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T21:09:08.767 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:09:08.767 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:09:08.768 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-26T21:09:08.768 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T21:09:08.768 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T21:09:08.768 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T21:09:08.768 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-26T21:09:08.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:09:08.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:08.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:08.977 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-26T21:09:09.027 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T21:09:09.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:09.028 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-04-26T21:09:09.078 INFO:teuthology.orchestra.run.smithi086.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-26T21:09:09.079 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:09:09.079 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-26T21:09:09.079 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:09:09.079 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-26T21:09:09.079 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-26T21:09:09.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:09:09.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:09.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:09.263 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T21:09:09.340 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:09:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:09:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:09:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-26T21:09:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:09.340 INFO:teuthology.orchestra.run.smithi086.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:09:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether ba:37:c8:59:28:e5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T21:09:09.340 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T21:09:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::b837:c8ff:fe59:28e5/64 scope link 2024-04-26T21:09:09.341 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:09.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:09.344 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T21:09:09.424 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:09:09.425 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:09:09.425 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:09:09.425 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:09.425 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-26T21:09:09.425 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:09.425 INFO:teuthology.orchestra.run.smithi086.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:09:09.425 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether ba:37:c8:59:28:e5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T21:09:09.425 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T21:09:09.425 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:09.425 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::b837:c8ff:fe59:28e5/64 scope link 2024-04-26T21:09:09.425 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:09.428 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-26T21:09:09.429 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:09:09.429 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-26T21:09:09.429 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-26T21:09:09.429 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T21:09:09.429 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-26T21:09:09.429 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-26T21:09:09.429 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-26T21:09:09.429 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-26T21:09:09.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:09:09.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:09.705 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:09:09.705 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-26T21:09:09.705 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.1 up 2024-04-26T21:09:09.705 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T21:09:09.705 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-26T21:09:09.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:09:09.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:09.908 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:09:09.909 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-26T21:09:09.951 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-26T21:09:09.952 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:09:09.952 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-26T21:09:10.044 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-04-26T21:09:10.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:10.044 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:09:10.169 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:09:10.169 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:09:10.207 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:09:10.207 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:09:10.207 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-26T21:09:10.207 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with new device syntax: 3@eaf60df9-83a0-4283-ab2c-83d2c02a2067.cephfs=/ 2024-04-26T21:09:10.208 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.26:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-26T21:09:10.209 INFO:teuthology.orchestra.run.smithi086.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-26T21:09:10.209 INFO:teuthology.orchestra.run.smithi086.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:09:10.209 INFO:teuthology.orchestra.run.smithi086.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:09:10.209 INFO:teuthology.orchestra.run.smithi086.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:09:10.209 INFO:teuthology.orchestra.run.smithi086.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:09:10.216 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:09:10.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:10.216 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-26T21:09:10.288 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:09:10.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:09:10.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:10.394 INFO:teuthology.orchestra.run.smithi086.stdout:client4436 2024-04-26T21:09:10.394 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:09:10.395 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:09:10.447 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:09:10.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:09:10.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:10.561 INFO:teuthology.orchestra.run.smithi086.stdout:client4436 2024-04-26T21:09:10.561 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:09:10.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:09:10.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:10.674 INFO:teuthology.orchestra.run.smithi086.stdout:client4436 2024-04-26T21:09:10.674 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eaf60df9-83a0-4283-ab2c-83d2c02a2067.client4436/status') 2024-04-26T21:09:10.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:09:10.800 INFO:teuthology.orchestra.run.smithi086.stdout:instance: client.4436 (3)192.168.0.2:0/38195011 2024-04-26T21:09:10.800 INFO:teuthology.orchestra.run.smithi086.stdout:blocklisted: false 2024-04-26T21:09:10.800 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records in 2024-04-26T21:09:10.800 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records out 2024-04-26T21:09:10.801 INFO:teuthology.orchestra.run.smithi086.stderr:67 bytes copied, 0.000113676 s, 589 kB/s 2024-04-26T21:09:10.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-26T21:09:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:10.813 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-26T21:09:10.829 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:09:11.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.052+0000 7fb326000640 1 Processor -- start 2024-04-26T21:09:11.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.053+0000 7fb326000640 1 -- start start 2024-04-26T21:09:11.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.054+0000 7fb326000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb3200d1c80 0x7fb3200d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.054+0000 7fb326000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3200d2060 0x7fb320162490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.054+0000 7fb326000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3201629d0 0x7fb3200c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.054+0000 7fb326000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fb3200b5f70 con 0x7fb3200d2060 2024-04-26T21:09:11.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.054+0000 7fb326000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb3200d3830 con 0x7fb3200d1c80 2024-04-26T21:09:11.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.054+0000 7fb326000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb320162e60 con 0x7fb3201629d0 2024-04-26T21:09:11.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.054+0000 7fb31fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3200d2060 0x7fb320162490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.055+0000 7fb31fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3200d2060 0x7fb320162490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46612/0 (socket says 172.21.15.26:46612) 2024-04-26T21:09:11.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.055+0000 7fb31fe00640 1 -- 172.21.15.26:0/1081501995 learned_addr learned my addr 172.21.15.26:0/1081501995 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:11.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.055+0000 7fb325600640 1 --2- 172.21.15.26:0/1081501995 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3201629d0 0x7fb3200c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.055+0000 7fb31fe00640 1 -- 172.21.15.26:0/1081501995 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3201629d0 msgr2=0x7fb3200c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:11.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.055+0000 7fb31fe00640 1 --2- 172.21.15.26:0/1081501995 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3201629d0 0x7fb3200c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.055+0000 7fb324c00640 1 --2- 172.21.15.26:0/1081501995 >> v2:172.21.15.86:3300/0 conn(0x7fb3200d1c80 0x7fb3200d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.055+0000 7fb31fe00640 1 -- 172.21.15.26:0/1081501995 >> v2:172.21.15.86:3300/0 conn(0x7fb3200d1c80 msgr2=0x7fb3200d3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:11.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.055+0000 7fb31fe00640 1 --2- 172.21.15.26:0/1081501995 >> v2:172.21.15.86:3300/0 conn(0x7fb3200d1c80 0x7fb3200d3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.055+0000 7fb31fe00640 1 -- 172.21.15.26:0/1081501995 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3200b61c0 con 0x7fb3200d2060 2024-04-26T21:09:11.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.056+0000 7fb31fe00640 1 --2- 172.21.15.26:0/1081501995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3200d2060 0x7fb320162490 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb32008d3b0 tx=0x7fb30c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3962a0aa8eba6ef2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:11.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.057+0000 7fb31f400640 1 -- 172.21.15.26:0/1081501995 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c037070 con 0x7fb3200d2060 2024-04-26T21:09:11.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.057+0000 7fb31f400640 1 -- 172.21.15.26:0/1081501995 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb30c032440 con 0x7fb3200d2060 2024-04-26T21:09:11.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.057+0000 7fb31f400640 1 -- 172.21.15.26:0/1081501995 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c032810 con 0x7fb3200d2060 2024-04-26T21:09:11.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.057+0000 7fb326000640 1 -- 172.21.15.26:0/1081501995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3200d2060 msgr2=0x7fb320162490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.057+0000 7fb326000640 1 --2- 172.21.15.26:0/1081501995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3200d2060 0x7fb320162490 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb32008d3b0 tx=0x7fb30c02a140 comp rx=0 tx=0).stop 2024-04-26T21:09:11.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.057+0000 7fb326000640 1 -- 172.21.15.26:0/1081501995 shutdown_connections 2024-04-26T21:09:11.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.057+0000 7fb326000640 1 --2- 172.21.15.26:0/1081501995 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3201629d0 0x7fb3200c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.057+0000 7fb326000640 1 --2- 172.21.15.26:0/1081501995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3200d2060 0x7fb320162490 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.057+0000 7fb326000640 1 --2- 172.21.15.26:0/1081501995 >> v2:172.21.15.86:3300/0 conn(0x7fb3200d1c80 0x7fb3200d3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.057+0000 7fb326000640 1 -- 172.21.15.26:0/1081501995 >> 172.21.15.26:0/1081501995 conn(0x7fb3200d5020 msgr2=0x7fb3200d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:11.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.057+0000 7fb326000640 1 -- 172.21.15.26:0/1081501995 shutdown_connections 2024-04-26T21:09:11.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.057+0000 7fb326000640 1 -- 172.21.15.26:0/1081501995 wait complete. 2024-04-26T21:09:11.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.059+0000 7fb326000640 1 Processor -- start 2024-04-26T21:09:11.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.059+0000 7fb326000640 1 -- start start 2024-04-26T21:09:11.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.060+0000 7fb326000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3200d1c80 0x7fb320153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.060+0000 7fb326000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3200d2060 0x7fb3200cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.060+0000 7fb326000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3201629d0 0x7fb3200ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.060+0000 7fb326000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb320074790 con 0x7fb3200d1c80 2024-04-26T21:09:11.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.060+0000 7fb326000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb320074610 con 0x7fb3200d2060 2024-04-26T21:09:11.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.060+0000 7fb326000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb3200d5b40 con 0x7fb3201629d0 2024-04-26T21:09:11.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.061+0000 7fb324c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3200d1c80 0x7fb320153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.061+0000 7fb324c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3200d1c80 0x7fb320153170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46628/0 (socket says 172.21.15.26:46628) 2024-04-26T21:09:11.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.061+0000 7fb324c00640 1 -- 172.21.15.26:0/1306785903 learned_addr learned my addr 172.21.15.26:0/1306785903 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.061+0000 7fb325600640 1 --2- 172.21.15.26:0/1306785903 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3201629d0 0x7fb3200ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.061+0000 7fb31fe00640 1 --2- 172.21.15.26:0/1306785903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3200d2060 0x7fb3200cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.061+0000 7fb324c00640 1 -- 172.21.15.26:0/1306785903 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3201629d0 msgr2=0x7fb3200ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.061+0000 7fb324c00640 1 --2- 172.21.15.26:0/1306785903 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3201629d0 0x7fb3200ce5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.061+0000 7fb324c00640 1 -- 172.21.15.26:0/1306785903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3200d2060 msgr2=0x7fb3200cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.061+0000 7fb324c00640 1 --2- 172.21.15.26:0/1306785903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3200d2060 0x7fb3200cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.061+0000 7fb324c00640 1 -- 172.21.15.26:0/1306785903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3200c8a80 con 0x7fb3200d1c80 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.061+0000 7fb324c00640 1 --2- 172.21.15.26:0/1306785903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3200d1c80 0x7fb320153170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb31400c7f0 tx=0x7fb31400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.062+0000 7fb31cc00640 1 -- 172.21.15.26:0/1306785903 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb314015070 con 0x7fb3200d1c80 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.062+0000 7fb31cc00640 1 -- 172.21.15.26:0/1306785903 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb314004220 con 0x7fb3200d1c80 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.062+0000 7fb31cc00640 1 -- 172.21.15.26:0/1306785903 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb314007620 con 0x7fb3200d1c80 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.062+0000 7fb326000640 1 -- 172.21.15.26:0/1306785903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3200b8fc0 con 0x7fb3200d1c80 2024-04-26T21:09:11.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.062+0000 7fb326000640 1 -- 172.21.15.26:0/1306785903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3200ccc30 con 0x7fb3200d1c80 2024-04-26T21:09:11.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.063+0000 7fb326000640 1 -- 172.21.15.26:0/1306785903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb32006a700 con 0x7fb3200d1c80 2024-04-26T21:09:11.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.064+0000 7fb31cc00640 1 -- 172.21.15.26:0/1306785903 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb3140079d0 con 0x7fb3200d1c80 2024-04-26T21:09:11.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.065+0000 7fb31cc00640 1 --2- 172.21.15.26:0/1306785903 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb30006c8f0 0x7fb30006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.066+0000 7fb31cc00640 1 -- 172.21.15.26:0/1306785903 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb31408be80 con 0x7fb3200d1c80 2024-04-26T21:09:11.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.066+0000 7fb31fe00640 1 --2- 172.21.15.26:0/1306785903 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb30006c8f0 0x7fb30006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.066+0000 7fb31fe00640 1 --2- 172.21.15.26:0/1306785903 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb30006c8f0 0x7fb30006edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb30c034000 tx=0x7fb30c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:11.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.070+0000 7fb31cc00640 1 -- 172.21.15.26:0/1306785903 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb3200c8a80 con 0x7fb3200d1c80 2024-04-26T21:09:11.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.274+0000 7fb326000640 1 -- 172.21.15.26:0/1306785903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb320069ae0 con 0x7fb3200d1c80 2024-04-26T21:09:11.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.277+0000 7fb31cc00640 1 -- 172.21.15.26:0/1306785903 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16557 (secure 0 0 0) 0x7fb31405a2a0 con 0x7fb3200d1c80 2024-04-26T21:09:11.279 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:11.279 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:08:39.177854+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:08:35.481169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:08:36.609498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:09:11.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.279+0000 7fb326000640 1 -- 172.21.15.26:0/1306785903 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb30006c8f0 msgr2=0x7fb30006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.279+0000 7fb326000640 1 --2- 172.21.15.26:0/1306785903 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb30006c8f0 0x7fb30006edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb30c034000 tx=0x7fb30c02e000 comp rx=0 tx=0).stop 2024-04-26T21:09:11.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.279+0000 7fb326000640 1 -- 172.21.15.26:0/1306785903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3200d1c80 msgr2=0x7fb320153170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.279+0000 7fb326000640 1 --2- 172.21.15.26:0/1306785903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3200d1c80 0x7fb320153170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb31400c7f0 tx=0x7fb31400ccb0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.280+0000 7fb326000640 1 -- 172.21.15.26:0/1306785903 shutdown_connections 2024-04-26T21:09:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.280+0000 7fb326000640 1 --2- 172.21.15.26:0/1306785903 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb30006c8f0 0x7fb30006edb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.280+0000 7fb326000640 1 --2- 172.21.15.26:0/1306785903 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3201629d0 0x7fb3200ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.280+0000 7fb326000640 1 --2- 172.21.15.26:0/1306785903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3200d2060 0x7fb3200cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.280+0000 7fb326000640 1 --2- 172.21.15.26:0/1306785903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3200d1c80 0x7fb320153170 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.280+0000 7fb326000640 1 -- 172.21.15.26:0/1306785903 >> 172.21.15.26:0/1306785903 conn(0x7fb3200d5020 msgr2=0x7fb32015faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.280+0000 7fb326000640 1 -- 172.21.15.26:0/1306785903 shutdown_connections 2024-04-26T21:09:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.280+0000 7fb326000640 1 -- 172.21.15.26:0/1306785903 wait complete. 2024-04-26T21:09:11.302 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:08:31.322511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:08:35.481169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:08:36.609498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:09:11.302 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:09:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.512+0000 7f6458e00640 1 Processor -- start 2024-04-26T21:09:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.512+0000 7f6458e00640 1 -- start start 2024-04-26T21:09:11.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.513+0000 7f6458e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f645414cf10 0x7f64540b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.513+0000 7f6458e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f64540726d0 0x7f64540cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.514+0000 7f6458e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f645406e2b0 con 0x7f64540cf5c0 2024-04-26T21:09:11.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.514+0000 7f6458e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f645406e550 con 0x7f64540726d0 2024-04-26T21:09:11.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.514+0000 7f6458e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f645406e830 con 0x7f645414cf10 2024-04-26T21:09:11.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.514+0000 7f6453e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f64540726d0 0x7f64540cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.514+0000 7f6453e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f64540726d0 0x7f64540cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46636/0 (socket says 172.21.15.26:46636) 2024-04-26T21:09:11.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.514+0000 7f6453e00640 1 -- 172.21.15.26:0/2898507693 learned_addr learned my addr 172.21.15.26:0/2898507693 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:11.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.514+0000 7f6453400640 1 --2- 172.21.15.26:0/2898507693 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f645414cf10 0x7f64540b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.514+0000 7f6453e00640 1 -- 172.21.15.26:0/2898507693 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f645414cf10 msgr2=0x7f64540b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:11.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.514+0000 7f6453e00640 1 --2- 172.21.15.26:0/2898507693 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f645414cf10 0x7f64540b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.514+0000 7f6453e00640 1 -- 172.21.15.26:0/2898507693 >> v1:172.21.15.86:6789/0 conn(0x7f64540cf5c0 legacy=0x7f6454071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:11.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.514+0000 7f6453e00640 1 -- 172.21.15.26:0/2898507693 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645406eb10 con 0x7f64540726d0 2024-04-26T21:09:11.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.516+0000 7f6453e00640 1 --2- 172.21.15.26:0/2898507693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64540726d0 0x7f64540cfa20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f644c009870 tx=0x7f644c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2776fe5ccc8f8804 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:11.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.516+0000 7f6452a00640 1 -- 172.21.15.26:0/2898507693 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c037070 con 0x7f64540726d0 2024-04-26T21:09:11.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.516+0000 7f6452a00640 1 -- 172.21.15.26:0/2898507693 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f644c032440 con 0x7f64540726d0 2024-04-26T21:09:11.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.516+0000 7f6452a00640 1 -- 172.21.15.26:0/2898507693 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c032810 con 0x7f64540726d0 2024-04-26T21:09:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.517+0000 7f6458e00640 1 -- 172.21.15.26:0/2898507693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64540726d0 msgr2=0x7f64540cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.517+0000 7f6458e00640 1 --2- 172.21.15.26:0/2898507693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64540726d0 0x7f64540cfa20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f644c009870 tx=0x7f644c02a210 comp rx=0 tx=0).stop 2024-04-26T21:09:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.517+0000 7f6458e00640 1 -- 172.21.15.26:0/2898507693 shutdown_connections 2024-04-26T21:09:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.517+0000 7f6458e00640 1 --2- 172.21.15.26:0/2898507693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64540726d0 0x7f64540cfa20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.517+0000 7f6458e00640 1 --2- 172.21.15.26:0/2898507693 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f645414cf10 0x7f64540b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.517+0000 7f6458e00640 1 -- 172.21.15.26:0/2898507693 >> 172.21.15.26:0/2898507693 conn(0x7f64540b2a10 msgr2=0x7f64540b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.517+0000 7f6458e00640 1 -- 172.21.15.26:0/2898507693 shutdown_connections 2024-04-26T21:09:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.517+0000 7f6458e00640 1 -- 172.21.15.26:0/2898507693 wait complete. 2024-04-26T21:09:11.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.519+0000 7f6458e00640 1 Processor -- start 2024-04-26T21:09:11.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.519+0000 7f6458e00640 1 -- start start 2024-04-26T21:09:11.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.520+0000 7f6458e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64540726d0 0x7f6454141310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.520+0000 7f6458e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64540cf5c0 0x7f6454146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.521+0000 7f6458e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f645414cf10 0x7f64541439a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.521+0000 7f6458e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f64540743f0 con 0x7f64540cf5c0 2024-04-26T21:09:11.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.521+0000 7f6458e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6454074270 con 0x7f64540726d0 2024-04-26T21:09:11.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.521+0000 7f6458e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f645414d2f0 con 0x7f645414cf10 2024-04-26T21:09:11.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.521+0000 7f644aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64540cf5c0 0x7f6454146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.521+0000 7f644aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64540cf5c0 0x7f6454146170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46638/0 (socket says 172.21.15.26:46638) 2024-04-26T21:09:11.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.521+0000 7f644aa00640 1 -- 172.21.15.26:0/1450781857 learned_addr learned my addr 172.21.15.26:0/1450781857 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:11.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.521+0000 7f644aa00640 1 -- 172.21.15.26:0/1450781857 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f645414cf10 msgr2=0x7f64541439a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.521+0000 7f6453e00640 1 --2- 172.21.15.26:0/1450781857 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f645414cf10 0x7f64541439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.521+0000 7f6453400640 1 --2- 172.21.15.26:0/1450781857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64540726d0 0x7f6454141310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.521+0000 7f644aa00640 1 --2- 172.21.15.26:0/1450781857 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f645414cf10 0x7f64541439a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.521+0000 7f644aa00640 1 -- 172.21.15.26:0/1450781857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64540726d0 msgr2=0x7f6454141310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.521+0000 7f644aa00640 1 --2- 172.21.15.26:0/1450781857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64540726d0 0x7f6454141310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.521+0000 7f644aa00640 1 -- 172.21.15.26:0/1450781857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64541450d0 con 0x7f64540cf5c0 2024-04-26T21:09:11.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.521+0000 7f6453400640 1 --2- 172.21.15.26:0/1450781857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64540726d0 0x7f6454141310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:11.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.522+0000 7f644aa00640 1 --2- 172.21.15.26:0/1450781857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64540cf5c0 0x7f6454146170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f643c00c460 tx=0x7f643c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:11.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.522+0000 7f644be00640 1 -- 172.21.15.26:0/1450781857 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c016020 con 0x7f64540cf5c0 2024-04-26T21:09:11.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.522+0000 7f644be00640 1 -- 172.21.15.26:0/1450781857 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f643c012870 con 0x7f64540cf5c0 2024-04-26T21:09:11.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.522+0000 7f644be00640 1 -- 172.21.15.26:0/1450781857 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c012c60 con 0x7f64540cf5c0 2024-04-26T21:09:11.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.522+0000 7f6458e00640 1 -- 172.21.15.26:0/1450781857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64541453b0 con 0x7f64540cf5c0 2024-04-26T21:09:11.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.522+0000 7f6458e00640 1 -- 172.21.15.26:0/1450781857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6454145970 con 0x7f64540cf5c0 2024-04-26T21:09:11.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.523+0000 7f6458e00640 1 -- 172.21.15.26:0/1450781857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f645406a700 con 0x7f64540cf5c0 2024-04-26T21:09:11.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.524+0000 7f644be00640 1 -- 172.21.15.26:0/1450781857 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f643c019020 con 0x7f64540cf5c0 2024-04-26T21:09:11.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.526+0000 7f644be00640 1 --2- 172.21.15.26:0/1450781857 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f643006c8f0 0x7f643006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.526+0000 7f644be00640 1 -- 172.21.15.26:0/1450781857 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f643c08b8a0 con 0x7f64540cf5c0 2024-04-26T21:09:11.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.526+0000 7f6453400640 1 --2- 172.21.15.26:0/1450781857 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f643006c8f0 0x7f643006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.527+0000 7f6453400640 1 --2- 172.21.15.26:0/1450781857 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f643006c8f0 0x7f643006edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6444007f50 tx=0x7f64440074a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:11.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.531+0000 7f644be00640 1 -- 172.21.15.26:0/1450781857 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f643c020030 con 0x7f64540cf5c0 2024-04-26T21:09:11.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.734+0000 7f6458e00640 1 -- 172.21.15.26:0/1450781857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f645406ffb0 con 0x7f64540cf5c0 2024-04-26T21:09:11.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.735+0000 7f644be00640 1 -- 172.21.15.26:0/1450781857 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f643c059cc0 con 0x7f64540cf5c0 2024-04-26T21:09:11.737 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-26T21:09:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.737+0000 7f6458e00640 1 -- 172.21.15.26:0/1450781857 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f643006c8f0 msgr2=0x7f643006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.737+0000 7f6458e00640 1 --2- 172.21.15.26:0/1450781857 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f643006c8f0 0x7f643006edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6444007f50 tx=0x7f64440074a0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.737+0000 7f6458e00640 1 -- 172.21.15.26:0/1450781857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64540cf5c0 msgr2=0x7f6454146170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.737+0000 7f6458e00640 1 --2- 172.21.15.26:0/1450781857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64540cf5c0 0x7f6454146170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f643c00c460 tx=0x7f643c00c920 comp rx=0 tx=0).stop 2024-04-26T21:09:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.737+0000 7f6458e00640 1 -- 172.21.15.26:0/1450781857 shutdown_connections 2024-04-26T21:09:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.737+0000 7f6458e00640 1 --2- 172.21.15.26:0/1450781857 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f643006c8f0 0x7f643006edb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.737+0000 7f6458e00640 1 --2- 172.21.15.26:0/1450781857 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f645414cf10 0x7f64541439a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.737+0000 7f6458e00640 1 --2- 172.21.15.26:0/1450781857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64540cf5c0 0x7f6454146170 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.737+0000 7f6458e00640 1 --2- 172.21.15.26:0/1450781857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64540726d0 0x7f6454141310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.738+0000 7f6458e00640 1 -- 172.21.15.26:0/1450781857 >> 172.21.15.26:0/1450781857 conn(0x7f64540b2a10 msgr2=0x7f64540b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.738+0000 7f6458e00640 1 -- 172.21.15.26:0/1450781857 shutdown_connections 2024-04-26T21:09:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.738+0000 7f6458e00640 1 -- 172.21.15.26:0/1450781857 wait complete. 2024-04-26T21:09:11.758 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:09:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.966+0000 7f17cbe00640 1 Processor -- start 2024-04-26T21:09:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.967+0000 7f17cbe00640 1 -- start start 2024-04-26T21:09:11.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.969+0000 7f17cbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f17cc0b6fe0 0x7f17cc0d2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.969+0000 7f17cbe00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17cc072830 con 0x7f17cc15ce30 2024-04-26T21:09:11.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.969+0000 7f17cbe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17cc0b5f40 con 0x7f17cc0d3470 2024-04-26T21:09:11.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.969+0000 7f17cbe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f17cc0b6220 con 0x7f17cc0b6fe0 2024-04-26T21:09:11.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.969+0000 7f17caa00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f17cc15ce30 0x7f17cc0d2150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45346/0 (socket says 172.21.15.26:45346) 2024-04-26T21:09:11.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.969+0000 7f17caa00640 1 -- 172.21.15.26:0/871838095 learned_addr learned my addr 172.21.15.26:0/871838095 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.969+0000 7f17ca000640 1 --2- 172.21.15.26:0/871838095 >> v2:172.21.15.86:3300/0 conn(0x7f17cc0b6fe0 0x7f17cc0d2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.969+0000 7f17c9600640 1 -- 172.21.15.26:0/871838095 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2005120195 0 0) 0x7f17bc001490 con 0x7f17cc15ce30 2024-04-26T21:09:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.970+0000 7f17c9600640 1 -- 172.21.15.26:0/871838095 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17b0003520 con 0x7f17cc15ce30 2024-04-26T21:09:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.970+0000 7f17c9600640 1 -- 172.21.15.26:0/871838095 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2262406467 0 0) 0x7f17b0003520 con 0x7f17cc15ce30 2024-04-26T21:09:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.970+0000 7f17c9600640 1 -- 172.21.15.26:0/871838095 >> v2:172.21.15.86:3300/0 conn(0x7f17cc0b6fe0 msgr2=0x7f17cc0d2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.970+0000 7f17c9600640 1 --2- 172.21.15.26:0/871838095 >> v2:172.21.15.86:3300/0 conn(0x7f17cc0b6fe0 0x7f17cc0d2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.970+0000 7f17c9600640 1 -- 172.21.15.26:0/871838095 >> v1:172.21.15.86:6789/0 conn(0x7f17cc0d3470 legacy=0x7f17cc15d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.970+0000 7f17c9600640 1 -- 172.21.15.26:0/871838095 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17cc0b6500 con 0x7f17cc15ce30 2024-04-26T21:09:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.970+0000 7f17c9600640 1 -- 172.21.15.26:0/871838095 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f17bc001bc0 con 0x7f17cc15ce30 2024-04-26T21:09:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.970+0000 7f17c9600640 1 -- 172.21.15.26:0/871838095 >> v1:172.21.15.26:6789/0 conn(0x7f17cc15ce30 legacy=0x7f17cc0d2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.970+0000 7f17c9600640 1 --2- 172.21.15.26:0/871838095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17b00039b0 0x7f17b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.970+0000 7f17c9600640 1 --2- 172.21.15.26:0/871838095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17b0004800 0x7f17b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.970+0000 7f17c9600640 1 --2- 172.21.15.26:0/871838095 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17b00051c0 0x7f17b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.970+0000 7f17c9600640 1 -- 172.21.15.26:0/871838095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc0b6220 con 0x7f17b00039b0 2024-04-26T21:09:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.970+0000 7f17c9600640 1 -- 172.21.15.26:0/871838095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17b000c160 con 0x7f17b0004800 2024-04-26T21:09:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.970+0000 7f17c9600640 1 -- 172.21.15.26:0/871838095 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f17b000c360 con 0x7f17b00051c0 2024-04-26T21:09:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.970+0000 7f17caa00640 1 --2- 172.21.15.26:0/871838095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17b00039b0 0x7f17b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.971+0000 7f17ca000640 1 --2- 172.21.15.26:0/871838095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17b0004800 0x7f17b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.971+0000 7f17caa00640 1 -- 172.21.15.26:0/871838095 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17b00051c0 msgr2=0x7f17b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.971+0000 7f17cb400640 1 --2- 172.21.15.26:0/871838095 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17b00051c0 0x7f17b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.971+0000 7f17caa00640 1 --2- 172.21.15.26:0/871838095 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17b00051c0 0x7f17b000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.971+0000 7f17caa00640 1 -- 172.21.15.26:0/871838095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17b0004800 msgr2=0x7f17b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.971+0000 7f17caa00640 1 --2- 172.21.15.26:0/871838095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17b0004800 0x7f17b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.971+0000 7f17caa00640 1 -- 172.21.15.26:0/871838095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b0003520 con 0x7f17b00039b0 2024-04-26T21:09:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.971+0000 7f17ca000640 1 --2- 172.21.15.26:0/871838095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17b0004800 0x7f17b0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.971+0000 7f17cb400640 1 -- 172.21.15.26:0/871838095 reap_dead start 2024-04-26T21:09:11.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.972+0000 7f17caa00640 1 --2- 172.21.15.26:0/871838095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17b00039b0 0x7f17b0003dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00aa20 tx=0x7f17b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feafeb8124612f51 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:11.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.972+0000 7f17c9600640 1 -- 172.21.15.26:0/871838095 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc00a640 con 0x7f17b00039b0 2024-04-26T21:09:11.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.972+0000 7f17c9600640 1 -- 172.21.15.26:0/871838095 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17bc02f070 con 0x7f17b00039b0 2024-04-26T21:09:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.972+0000 7f17c9600640 1 -- 172.21.15.26:0/871838095 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc034470 con 0x7f17b00039b0 2024-04-26T21:09:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.973+0000 7f17cbe00640 1 -- 172.21.15.26:0/871838095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17b00039b0 msgr2=0x7f17b0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.973+0000 7f17cbe00640 1 --2- 172.21.15.26:0/871838095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17b00039b0 0x7f17b0003dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00aa20 tx=0x7f17b0005830 comp rx=0 tx=0).stop 2024-04-26T21:09:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.973+0000 7f17cbe00640 1 -- 172.21.15.26:0/871838095 shutdown_connections 2024-04-26T21:09:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.973+0000 7f17cbe00640 1 --2- 172.21.15.26:0/871838095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17b00039b0 0x7f17b0003dd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.973+0000 7f17cbe00640 1 -- 172.21.15.26:0/871838095 >> 172.21.15.26:0/871838095 conn(0x7f17cc06c240 msgr2=0x7f17cc06c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.973+0000 7f17cbe00640 1 -- 172.21.15.26:0/871838095 shutdown_connections 2024-04-26T21:09:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.973+0000 7f17cbe00640 1 -- 172.21.15.26:0/871838095 wait complete. 2024-04-26T21:09:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.975+0000 7f17cbe00640 1 Processor -- start 2024-04-26T21:09:11.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.975+0000 7f17cbe00640 1 -- start start 2024-04-26T21:09:11.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.976+0000 7f17cbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17b00039b0 0x7f17cc1f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.976+0000 7f17cbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17cc1e7540 0x7f17cc1e79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.976+0000 7f17cbe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17cc15d210 0x7f17cc15d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.976+0000 7f17cbe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc0d2f30 con 0x7f17cc1e7540 2024-04-26T21:09:11.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.976+0000 7f17cbe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc0b6ac0 con 0x7f17b00039b0 2024-04-26T21:09:11.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.976+0000 7f17cbe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f17cc0d30b0 con 0x7f17cc15d210 2024-04-26T21:09:11.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.976+0000 7f17ca000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17cc1e7540 0x7f17cc1e79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.977+0000 7f17ca000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17cc1e7540 0x7f17cc1e79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46660/0 (socket says 172.21.15.26:46660) 2024-04-26T21:09:11.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.977+0000 7f17ca000640 1 -- 172.21.15.26:0/4024236372 learned_addr learned my addr 172.21.15.26:0/4024236372 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:11.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.977+0000 7f17caa00640 1 --2- 172.21.15.26:0/4024236372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17b00039b0 0x7f17cc1f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.977+0000 7f17cb400640 1 --2- 172.21.15.26:0/4024236372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17cc15d210 0x7f17cc15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.977+0000 7f17ca000640 1 -- 172.21.15.26:0/4024236372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17cc15d210 msgr2=0x7f17cc15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.977+0000 7f17ca000640 1 --2- 172.21.15.26:0/4024236372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17cc15d210 0x7f17cc15d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.977+0000 7f17ca000640 1 -- 172.21.15.26:0/4024236372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17b00039b0 msgr2=0x7f17cc1f7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:11.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.977+0000 7f17ca000640 1 --2- 172.21.15.26:0/4024236372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17b00039b0 0x7f17cc1f7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:11.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.977+0000 7f17ca000640 1 -- 172.21.15.26:0/4024236372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17cc15dbd0 con 0x7f17cc1e7540 2024-04-26T21:09:11.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.977+0000 7f17ca000640 1 --2- 172.21.15.26:0/4024236372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17cc1e7540 0x7f17cc1e79c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f17b40107c0 tx=0x7f17b4010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:11.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.978+0000 7f17c2a00640 1 -- 172.21.15.26:0/4024236372 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b400f040 con 0x7f17cc1e7540 2024-04-26T21:09:11.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.978+0000 7f17c2a00640 1 -- 172.21.15.26:0/4024236372 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17b400eb60 con 0x7f17cc1e7540 2024-04-26T21:09:11.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.978+0000 7f17c2a00640 1 -- 172.21.15.26:0/4024236372 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b401c400 con 0x7f17cc1e7540 2024-04-26T21:09:11.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.978+0000 7f17cbe00640 1 -- 172.21.15.26:0/4024236372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17cc15de60 con 0x7f17cc1e7540 2024-04-26T21:09:11.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.978+0000 7f17cbe00640 1 -- 172.21.15.26:0/4024236372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17cc15e420 con 0x7f17cc1e7540 2024-04-26T21:09:11.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.979+0000 7f17cbe00640 1 -- 172.21.15.26:0/4024236372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17cc0693d0 con 0x7f17cc1e7540 2024-04-26T21:09:11.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.980+0000 7f17c2a00640 1 -- 172.21.15.26:0/4024236372 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f17b401c5a0 con 0x7f17cc1e7540 2024-04-26T21:09:11.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.981+0000 7f17c2a00640 1 --2- 172.21.15.26:0/4024236372 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f17a806c920 0x7f17a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:11.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.981+0000 7f17c2a00640 1 -- 172.21.15.26:0/4024236372 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f17b408c0d0 con 0x7f17cc1e7540 2024-04-26T21:09:11.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.981+0000 7f17caa00640 1 --2- 172.21.15.26:0/4024236372 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f17a806c920 0x7f17a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:11.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.982+0000 7f17caa00640 1 --2- 172.21.15.26:0/4024236372 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f17a806c920 0x7f17a806ede0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f17bc033000 tx=0x7f17bc007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:11.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:11.985+0000 7f17c2a00640 1 -- 172.21.15.26:0/4024236372 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f17b4018030 con 0x7f17cc1e7540 2024-04-26T21:09:12.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.192+0000 7f17cbe00640 1 -- 172.21.15.26:0/4024236372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f17cc160830 con 0x7f17cc1e7540 2024-04-26T21:09:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.192+0000 7f17c2a00640 1 -- 172.21.15.26:0/4024236372 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f17b405a460 con 0x7f17cc1e7540 2024-04-26T21:09:12.194 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-26T21:09:12.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.195+0000 7f17cbe00640 1 -- 172.21.15.26:0/4024236372 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f17a806c920 msgr2=0x7f17a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:12.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.195+0000 7f17cbe00640 1 --2- 172.21.15.26:0/4024236372 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f17a806c920 0x7f17a806ede0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f17bc033000 tx=0x7f17bc007000 comp rx=0 tx=0).stop 2024-04-26T21:09:12.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.195+0000 7f17cbe00640 1 -- 172.21.15.26:0/4024236372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17cc1e7540 msgr2=0x7f17cc1e79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:12.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.195+0000 7f17cbe00640 1 --2- 172.21.15.26:0/4024236372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17cc1e7540 0x7f17cc1e79c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f17b40107c0 tx=0x7f17b4010c80 comp rx=0 tx=0).stop 2024-04-26T21:09:12.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.195+0000 7f17cbe00640 1 -- 172.21.15.26:0/4024236372 shutdown_connections 2024-04-26T21:09:12.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.195+0000 7f17cbe00640 1 --2- 172.21.15.26:0/4024236372 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f17a806c920 0x7f17a806ede0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:12.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.195+0000 7f17cbe00640 1 --2- 172.21.15.26:0/4024236372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17cc15d210 0x7f17cc15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:12.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.195+0000 7f17cbe00640 1 --2- 172.21.15.26:0/4024236372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17cc1e7540 0x7f17cc1e79c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:12.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.196+0000 7f17cbe00640 1 --2- 172.21.15.26:0/4024236372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17b00039b0 0x7f17cc1f7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:12.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.196+0000 7f17cbe00640 1 -- 172.21.15.26:0/4024236372 >> 172.21.15.26:0/4024236372 conn(0x7f17cc06c240 msgr2=0x7f17cc06c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:12.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.196+0000 7f17cbe00640 1 -- 172.21.15.26:0/4024236372 shutdown_connections 2024-04-26T21:09:12.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.196+0000 7f17cbe00640 1 -- 172.21.15.26:0/4024236372 wait complete. 2024-04-26T21:09:12.217 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:09:12.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.418+0000 7f9f8e400640 1 Processor -- start 2024-04-26T21:09:12.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.419+0000 7f9f8e400640 1 -- start start 2024-04-26T21:09:12.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.420+0000 7f9f8e400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f8815f690 0x7f9f880d1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:12.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.420+0000 7f9f8e400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9f880d3770 0x7f9f880d1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:12.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.420+0000 7f9f8e400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f880d3d10 con 0x7f9f8815f2b0 2024-04-26T21:09:12.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.420+0000 7f9f8e400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f9f880d3ff0 con 0x7f9f880d3770 2024-04-26T21:09:12.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.420+0000 7f9f8e400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9f880b8190 con 0x7f9f8815f690 2024-04-26T21:09:12.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.421+0000 7f9f87e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9f880d3770 0x7f9f880d1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:12.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.421+0000 7f9f87e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9f880d3770 0x7f9f880d1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46672/0 (socket says 172.21.15.26:46672) 2024-04-26T21:09:12.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.421+0000 7f9f87e00640 1 -- 172.21.15.26:0/58433590 learned_addr learned my addr 172.21.15.26:0/58433590 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:12.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.421+0000 7f9f86a00640 1 --2- 172.21.15.26:0/58433590 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f8815f690 0x7f9f880d1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:12.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.421+0000 7f9f87e00640 1 -- 172.21.15.26:0/58433590 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f8815f690 msgr2=0x7f9f880d1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:12.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.421+0000 7f9f87e00640 1 --2- 172.21.15.26:0/58433590 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f8815f690 0x7f9f880d1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:12.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.421+0000 7f9f87e00640 1 -- 172.21.15.26:0/58433590 >> v1:172.21.15.86:6789/0 conn(0x7f9f8815f2b0 legacy=0x7f9f880d5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:12.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.421+0000 7f9f87e00640 1 -- 172.21.15.26:0/58433590 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f880b8470 con 0x7f9f880d3770 2024-04-26T21:09:12.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.423+0000 7f9f87e00640 1 --2- 172.21.15.26:0/58433590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f880d3770 0x7f9f880d1fd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009e50 tx=0x7f9f8002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39abfb552d5425ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:12.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.423+0000 7f9f86000640 1 -- 172.21.15.26:0/58433590 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80036070 con 0x7f9f880d3770 2024-04-26T21:09:12.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.424+0000 7f9f86000640 1 -- 172.21.15.26:0/58433590 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f80031600 con 0x7f9f880d3770 2024-04-26T21:09:12.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.424+0000 7f9f86000640 1 -- 172.21.15.26:0/58433590 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f800319d0 con 0x7f9f880d3770 2024-04-26T21:09:12.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.424+0000 7f9f8e400640 1 -- 172.21.15.26:0/58433590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f880d3770 msgr2=0x7f9f880d1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:12.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.424+0000 7f9f8e400640 1 --2- 172.21.15.26:0/58433590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f880d3770 0x7f9f880d1fd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009e50 tx=0x7f9f8002a680 comp rx=0 tx=0).stop 2024-04-26T21:09:12.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.425+0000 7f9f8e400640 1 -- 172.21.15.26:0/58433590 shutdown_connections 2024-04-26T21:09:12.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.425+0000 7f9f8e400640 1 --2- 172.21.15.26:0/58433590 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f8815f690 0x7f9f880d1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:12.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.425+0000 7f9f8e400640 1 --2- 172.21.15.26:0/58433590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f880d3770 0x7f9f880d1fd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:12.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.425+0000 7f9f8e400640 1 -- 172.21.15.26:0/58433590 >> 172.21.15.26:0/58433590 conn(0x7f9f8809e490 msgr2=0x7f9f880d51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:12.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.425+0000 7f9f8e400640 1 -- 172.21.15.26:0/58433590 shutdown_connections 2024-04-26T21:09:12.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.425+0000 7f9f8e400640 1 -- 172.21.15.26:0/58433590 wait complete. 2024-04-26T21:09:12.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.427+0000 7f9f8e400640 1 Processor -- start 2024-04-26T21:09:12.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.427+0000 7f9f8e400640 1 -- start start 2024-04-26T21:09:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.428+0000 7f9f8e400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f880d3770 0x7f9f880ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.428+0000 7f9f8e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8815f2b0 0x7f9f8815cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.428+0000 7f9f8e400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f8815f690 0x7f9f8814c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.428+0000 7f9f8e400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9f880b8d60 con 0x7f9f8815f690 2024-04-26T21:09:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.428+0000 7f9f8e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9f880b8be0 con 0x7f9f8815f2b0 2024-04-26T21:09:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.428+0000 7f9f8e400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9f880b8ee0 con 0x7f9f880d3770 2024-04-26T21:09:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.428+0000 7f9f87e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f8815f690 0x7f9f8814c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.428+0000 7f9f87e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f8815f690 0x7f9f8814c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46674/0 (socket says 172.21.15.26:46674) 2024-04-26T21:09:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.428+0000 7f9f87e00640 1 -- 172.21.15.26:0/2239630561 learned_addr learned my addr 172.21.15.26:0/2239630561 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.428+0000 7f9f86a00640 1 --2- 172.21.15.26:0/2239630561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8815f2b0 0x7f9f8815cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.428+0000 7f9f87400640 1 --2- 172.21.15.26:0/2239630561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f880d3770 0x7f9f880ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.429+0000 7f9f87e00640 1 -- 172.21.15.26:0/2239630561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f880d3770 msgr2=0x7f9f880ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.429+0000 7f9f87e00640 1 --2- 172.21.15.26:0/2239630561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f880d3770 0x7f9f880ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.429+0000 7f9f87e00640 1 -- 172.21.15.26:0/2239630561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8815f2b0 msgr2=0x7f9f8815cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.429+0000 7f9f87e00640 1 --2- 172.21.15.26:0/2239630561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8815f2b0 0x7f9f8815cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.429+0000 7f9f87e00640 1 -- 172.21.15.26:0/2239630561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f880caae0 con 0x7f9f8815f690 2024-04-26T21:09:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.429+0000 7f9f87e00640 1 --2- 172.21.15.26:0/2239630561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f8815f690 0x7f9f8814c0c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009e50 tx=0x7f9f80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.429+0000 7f9f7f400640 1 -- 172.21.15.26:0/2239630561 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80036050 con 0x7f9f8815f690 2024-04-26T21:09:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.429+0000 7f9f7f400640 1 -- 172.21.15.26:0/2239630561 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f80042070 con 0x7f9f8815f690 2024-04-26T21:09:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.429+0000 7f9f7f400640 1 -- 172.21.15.26:0/2239630561 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f80009510 con 0x7f9f8815f690 2024-04-26T21:09:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.429+0000 7f9f8e400640 1 -- 172.21.15.26:0/2239630561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f880cadc0 con 0x7f9f8815f690 2024-04-26T21:09:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.430+0000 7f9f8e400640 1 -- 172.21.15.26:0/2239630561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f880cb380 con 0x7f9f8815f690 2024-04-26T21:09:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.431+0000 7f9f8e400640 1 -- 172.21.15.26:0/2239630561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f88050730 con 0x7f9f8815f690 2024-04-26T21:09:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.432+0000 7f9f7f400640 1 -- 172.21.15.26:0/2239630561 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f80007bd0 con 0x7f9f8815f690 2024-04-26T21:09:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.432+0000 7f9f7f400640 1 --2- 172.21.15.26:0/2239630561 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f9f6806c8f0 0x7f9f6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.433+0000 7f9f7f400640 1 -- 172.21.15.26:0/2239630561 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9f800aca20 con 0x7f9f8815f690 2024-04-26T21:09:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.433+0000 7f9f87400640 1 --2- 172.21.15.26:0/2239630561 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f9f6806c8f0 0x7f9f6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.434+0000 7f9f87400640 1 --2- 172.21.15.26:0/2239630561 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f9f6806c8f0 0x7f9f6806edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9f78001640 tx=0x7f9f78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:12.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.437+0000 7f9f7f400640 1 -- 172.21.15.26:0/2239630561 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9f8007ae40 con 0x7f9f8815f690 2024-04-26T21:09:12.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.640+0000 7f9f8e400640 1 -- 172.21.15.26:0/2239630561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9f88162b60 con 0x7f9f8815f690 2024-04-26T21:09:12.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.641+0000 7f9f7f400640 1 -- 172.21.15.26:0/2239630561 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f9f8007a740 con 0x7f9f8815f690 2024-04-26T21:09:12.642 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-26T21:09:12.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.643+0000 7f9f8e400640 1 -- 172.21.15.26:0/2239630561 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f9f6806c8f0 msgr2=0x7f9f6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.643+0000 7f9f8e400640 1 --2- 172.21.15.26:0/2239630561 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f9f6806c8f0 0x7f9f6806edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9f78001640 tx=0x7f9f78009040 comp rx=0 tx=0).stop 2024-04-26T21:09:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.643+0000 7f9f8e400640 1 -- 172.21.15.26:0/2239630561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f8815f690 msgr2=0x7f9f8814c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.643+0000 7f9f8e400640 1 --2- 172.21.15.26:0/2239630561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f8815f690 0x7f9f8814c0c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009e50 tx=0x7f9f80004a40 comp rx=0 tx=0).stop 2024-04-26T21:09:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.643+0000 7f9f8e400640 1 -- 172.21.15.26:0/2239630561 shutdown_connections 2024-04-26T21:09:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.643+0000 7f9f8e400640 1 --2- 172.21.15.26:0/2239630561 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f9f6806c8f0 0x7f9f6806edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.643+0000 7f9f8e400640 1 --2- 172.21.15.26:0/2239630561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f8815f690 0x7f9f8814c0c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.643+0000 7f9f8e400640 1 --2- 172.21.15.26:0/2239630561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f8815f2b0 0x7f9f8815cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.643+0000 7f9f8e400640 1 --2- 172.21.15.26:0/2239630561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f880d3770 0x7f9f880ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.643+0000 7f9f8e400640 1 -- 172.21.15.26:0/2239630561 >> 172.21.15.26:0/2239630561 conn(0x7f9f8809e490 msgr2=0x7f9f8806cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.644+0000 7f9f8e400640 1 -- 172.21.15.26:0/2239630561 shutdown_connections 2024-04-26T21:09:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.644+0000 7f9f8e400640 1 -- 172.21.15.26:0/2239630561 wait complete. 2024-04-26T21:09:12.663 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:09:12.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.836+0000 7efdc6a00640 1 Processor -- start 2024-04-26T21:09:12.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.837+0000 7efdc6a00640 1 -- start start 2024-04-26T21:09:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.838+0000 7efdc6a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efdc0146990 0x7efdc00b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.838+0000 7efdc6a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efdc006ec20 0x7efdc00d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.838+0000 7efdc6a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdc00d1620 con 0x7efdc00d1210 2024-04-26T21:09:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.838+0000 7efdc6a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7efdc0146d70 con 0x7efdc006ec20 2024-04-26T21:09:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.838+0000 7efdc6a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7efdc00b4480 con 0x7efdc0146990 2024-04-26T21:09:12.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.838+0000 7efdc4a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efdc00d1210 0x7efdc0072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45356/0 (socket says 172.21.15.26:45356) 2024-04-26T21:09:12.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.838+0000 7efdc4a00640 1 -- 172.21.15.26:0/4018103982 learned_addr learned my addr 172.21.15.26:0/4018103982 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:12.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.838+0000 7efdbf400640 1 --2- 172.21.15.26:0/4018103982 >> v2:172.21.15.86:3300/0 conn(0x7efdc006ec20 0x7efdc00d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:12.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.838+0000 7efdbfe00640 1 --2- 172.21.15.26:0/4018103982 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efdc0146990 0x7efdc00b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:12.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.839+0000 7efdbea00640 1 -- 172.21.15.26:0/4018103982 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1408635346 0 0) 0x7efdb8001490 con 0x7efdc00d1210 2024-04-26T21:09:12.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.839+0000 7efdbea00640 1 -- 172.21.15.26:0/4018103982 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efda4003520 con 0x7efdc00d1210 2024-04-26T21:09:12.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.839+0000 7efdbea00640 1 -- 172.21.15.26:0/4018103982 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 902106225 0 0) 0x7efda4003520 con 0x7efdc00d1210 2024-04-26T21:09:12.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.839+0000 7efdbea00640 1 -- 172.21.15.26:0/4018103982 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efdc0146990 msgr2=0x7efdc00b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:12.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.839+0000 7efdbea00640 1 --2- 172.21.15.26:0/4018103982 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efdc0146990 0x7efdc00b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:12.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.839+0000 7efdbea00640 1 -- 172.21.15.26:0/4018103982 >> v2:172.21.15.86:3300/0 conn(0x7efdc006ec20 msgr2=0x7efdc00d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:12.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.839+0000 7efdbea00640 1 --2- 172.21.15.26:0/4018103982 >> v2:172.21.15.86:3300/0 conn(0x7efdc006ec20 0x7efdc00d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:12.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.839+0000 7efdbea00640 1 -- 172.21.15.26:0/4018103982 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc0073b70 con 0x7efdc00d1210 2024-04-26T21:09:12.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.839+0000 7efdbea00640 1 -- 172.21.15.26:0/4018103982 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7efdb8001bc0 con 0x7efdc00d1210 2024-04-26T21:09:12.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.839+0000 7efdbea00640 1 -- 172.21.15.26:0/4018103982 >> v1:172.21.15.26:6789/0 conn(0x7efdc00d1210 legacy=0x7efdc0072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:12.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.839+0000 7efdbea00640 1 --2- 172.21.15.26:0/4018103982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda40039b0 0x7efda4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:12.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.839+0000 7efdbea00640 1 --2- 172.21.15.26:0/4018103982 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efdc0146990 0x7efda4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:12.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.839+0000 7efdbea00640 1 --2- 172.21.15.26:0/4018103982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efda4004d40 0x7efda40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:12.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.839+0000 7efdbea00640 1 -- 172.21.15.26:0/4018103982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efdc0146d70 con 0x7efda40039b0 2024-04-26T21:09:12.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.839+0000 7efdbea00640 1 -- 172.21.15.26:0/4018103982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efdc00b4480 con 0x7efda4004d40 2024-04-26T21:09:12.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.839+0000 7efdbea00640 1 -- 172.21.15.26:0/4018103982 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7efda400b9c0 con 0x7efdc0146990 2024-04-26T21:09:12.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.839+0000 7efdbfe00640 1 --2- 172.21.15.26:0/4018103982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda40039b0 0x7efda4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:12.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.840+0000 7efdc4a00640 1 --2- 172.21.15.26:0/4018103982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efda4004d40 0x7efda40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:12.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.840+0000 7efdbfe00640 1 -- 172.21.15.26:0/4018103982 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efdc0146990 msgr2=0x7efda4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:12.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.840+0000 7efdbfe00640 1 --2- 172.21.15.26:0/4018103982 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efdc0146990 0x7efda4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:12.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.840+0000 7efdbfe00640 1 -- 172.21.15.26:0/4018103982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efda4004d40 msgr2=0x7efda40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:12.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.840+0000 7efdbfe00640 1 --2- 172.21.15.26:0/4018103982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efda4004d40 0x7efda40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:12.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.840+0000 7efdbfe00640 1 -- 172.21.15.26:0/4018103982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda4003520 con 0x7efda40039b0 2024-04-26T21:09:12.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.841+0000 7efdbfe00640 1 --2- 172.21.15.26:0/4018103982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda40039b0 0x7efda4003dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efdb0009ac0 tx=0x7efda4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b63d48680be382 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:12.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.841+0000 7efdbea00640 1 -- 172.21.15.26:0/4018103982 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0036070 con 0x7efda40039b0 2024-04-26T21:09:12.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.841+0000 7efdbea00640 1 -- 172.21.15.26:0/4018103982 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb0031400 con 0x7efda40039b0 2024-04-26T21:09:12.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.841+0000 7efdbea00640 1 -- 172.21.15.26:0/4018103982 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0009690 con 0x7efda40039b0 2024-04-26T21:09:12.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.842+0000 7efdc6a00640 1 -- 172.21.15.26:0/4018103982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda40039b0 msgr2=0x7efda4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:12.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.842+0000 7efdc6a00640 1 --2- 172.21.15.26:0/4018103982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efda40039b0 0x7efda4003dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efdb0009ac0 tx=0x7efda4005830 comp rx=0 tx=0).stop 2024-04-26T21:09:12.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.842+0000 7efdc4a00640 1 -- 172.21.15.26:0/4018103982 reap_dead start 2024-04-26T21:09:12.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.842+0000 7efdc6a00640 1 -- 172.21.15.26:0/4018103982 shutdown_connections 2024-04-26T21:09:12.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.842+0000 7efdc6a00640 1 -- 172.21.15.26:0/4018103982 >> 172.21.15.26:0/4018103982 conn(0x7efdc00cd840 msgr2=0x7efdc00cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:12.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.842+0000 7efdc6a00640 1 -- 172.21.15.26:0/4018103982 shutdown_connections 2024-04-26T21:09:12.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.842+0000 7efdc6a00640 1 -- 172.21.15.26:0/4018103982 wait complete. 2024-04-26T21:09:12.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.844+0000 7efdc6a00640 1 Processor -- start 2024-04-26T21:09:12.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.844+0000 7efdc6a00640 1 -- start start 2024-04-26T21:09:12.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.845+0000 7efdc6a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efda40039b0 0x7efdc01f7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:12.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.845+0000 7efdc6a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdc01559d0 0x7efdc0159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:12.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.845+0000 7efdc6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc0155e50 0x7efdc015a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:12.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.845+0000 7efdc6a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efdc006e420 con 0x7efdc01559d0 2024-04-26T21:09:12.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.845+0000 7efdc6a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efdc0074130 con 0x7efdc0155e50 2024-04-26T21:09:12.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.845+0000 7efdc6a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7efdc006e700 con 0x7efda40039b0 2024-04-26T21:09:12.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.845+0000 7efdbf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdc01559d0 0x7efdc0159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:12.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.845+0000 7efdbf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdc01559d0 0x7efdc0159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46690/0 (socket says 172.21.15.26:46690) 2024-04-26T21:09:12.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.845+0000 7efdbf400640 1 -- 172.21.15.26:0/1857556453 learned_addr learned my addr 172.21.15.26:0/1857556453 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:12.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.845+0000 7efdc4a00640 1 --2- 172.21.15.26:0/1857556453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc0155e50 0x7efdc015a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:12.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.845+0000 7efdbf400640 1 -- 172.21.15.26:0/1857556453 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efda40039b0 msgr2=0x7efdc01f7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:12.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.845+0000 7efdbf400640 1 --2- 172.21.15.26:0/1857556453 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efda40039b0 0x7efdc01f7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:12.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.845+0000 7efdbf400640 1 -- 172.21.15.26:0/1857556453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc0155e50 msgr2=0x7efdc015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:12.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.845+0000 7efdbf400640 1 --2- 172.21.15.26:0/1857556453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc0155e50 0x7efdc015a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:12.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.845+0000 7efdbf400640 1 -- 172.21.15.26:0/1857556453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc0156420 con 0x7efdc01559d0 2024-04-26T21:09:12.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.845+0000 7efdc4a00640 1 --2- 172.21.15.26:0/1857556453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc0155e50 0x7efdc015a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:12.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.846+0000 7efdbf400640 1 --2- 172.21.15.26:0/1857556453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdc01559d0 0x7efdc0159e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efdac00e8c0 tx=0x7efdac00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:12.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.846+0000 7efdb7e00640 1 -- 172.21.15.26:0/1857556453 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac017070 con 0x7efdc01559d0 2024-04-26T21:09:12.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.846+0000 7efdb7e00640 1 -- 172.21.15.26:0/1857556453 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdac00c8e0 con 0x7efdc01559d0 2024-04-26T21:09:12.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.846+0000 7efdb7e00640 1 -- 172.21.15.26:0/1857556453 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac00ccb0 con 0x7efdc01559d0 2024-04-26T21:09:12.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.846+0000 7efdc6a00640 1 -- 172.21.15.26:0/1857556453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc0156700 con 0x7efdc01559d0 2024-04-26T21:09:12.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.846+0000 7efdc6a00640 1 -- 172.21.15.26:0/1857556453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc0156cc0 con 0x7efdc01559d0 2024-04-26T21:09:12.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.848+0000 7efdc6a00640 1 -- 172.21.15.26:0/1857556453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdc006a700 con 0x7efdc01559d0 2024-04-26T21:09:12.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.848+0000 7efdb7e00640 1 -- 172.21.15.26:0/1857556453 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efdac01a070 con 0x7efdc01559d0 2024-04-26T21:09:12.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.849+0000 7efdb7e00640 1 --2- 172.21.15.26:0/1857556453 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7efda006c8f0 0x7efda006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:12.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.849+0000 7efdb7e00640 1 -- 172.21.15.26:0/1857556453 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efdac08bde0 con 0x7efdc01559d0 2024-04-26T21:09:12.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.849+0000 7efdbfe00640 1 --2- 172.21.15.26:0/1857556453 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7efda006c8f0 0x7efda006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:12.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.850+0000 7efdbfe00640 1 --2- 172.21.15.26:0/1857556453 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7efda006c8f0 0x7efda006edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efdb0033000 tx=0x7efdb002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:12.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:12.853+0000 7efdb7e00640 1 -- 172.21.15.26:0/1857556453 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efdac05a170 con 0x7efdc01559d0 2024-04-26T21:09:13.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.054+0000 7efdc6a00640 1 -- 172.21.15.26:0/1857556453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efdc006fd50 con 0x7efdc01559d0 2024-04-26T21:09:13.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.057+0000 7efdb7e00640 1 -- 172.21.15.26:0/1857556453 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16557 (secure 0 0 0) 0x7efdac059a70 con 0x7efdc01559d0 2024-04-26T21:09:13.059 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:13.060 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:08:39.177854+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:08:35.481169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:08:36.609498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:09:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.060+0000 7efdc6a00640 1 -- 172.21.15.26:0/1857556453 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7efda006c8f0 msgr2=0x7efda006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.060+0000 7efdc6a00640 1 --2- 172.21.15.26:0/1857556453 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7efda006c8f0 0x7efda006edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efdb0033000 tx=0x7efdb002e000 comp rx=0 tx=0).stop 2024-04-26T21:09:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.060+0000 7efdc6a00640 1 -- 172.21.15.26:0/1857556453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdc01559d0 msgr2=0x7efdc0159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.060+0000 7efdc6a00640 1 --2- 172.21.15.26:0/1857556453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdc01559d0 0x7efdc0159e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efdac00e8c0 tx=0x7efdac00ed80 comp rx=0 tx=0).stop 2024-04-26T21:09:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.060+0000 7efdc6a00640 1 -- 172.21.15.26:0/1857556453 shutdown_connections 2024-04-26T21:09:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.060+0000 7efdc6a00640 1 --2- 172.21.15.26:0/1857556453 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7efda006c8f0 0x7efda006edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.060+0000 7efdc6a00640 1 --2- 172.21.15.26:0/1857556453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efdc0155e50 0x7efdc015a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.060+0000 7efdc6a00640 1 --2- 172.21.15.26:0/1857556453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdc01559d0 0x7efdc0159e20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.060+0000 7efdc6a00640 1 --2- 172.21.15.26:0/1857556453 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efda40039b0 0x7efdc01f7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.060+0000 7efdc6a00640 1 -- 172.21.15.26:0/1857556453 >> 172.21.15.26:0/1857556453 conn(0x7efdc00cd840 msgr2=0x7efdc00b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.060+0000 7efdc6a00640 1 -- 172.21.15.26:0/1857556453 shutdown_connections 2024-04-26T21:09:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.060+0000 7efdc6a00640 1 -- 172.21.15.26:0/1857556453 wait complete. 2024-04-26T21:09:13.080 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:08:31.322511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:08:35.481169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:08:36.609498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:09:13.080 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:09:13.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.275+0000 7fe1c9a00640 1 Processor -- start 2024-04-26T21:09:13.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.276+0000 7fe1c9a00640 1 -- start start 2024-04-26T21:09:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.277+0000 7fe1c9a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe1c415ce40 0x7fe1c4162e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.277+0000 7fe1c9a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe1c40d0b50 0x7fe1c40d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.277+0000 7fe1c9a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe1c40b4aa0 0x7fe1c40b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.277+0000 7fe1c9a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fe1c40cf960 con 0x7fe1c40d0b50 2024-04-26T21:09:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.277+0000 7fe1c9a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe1c415d220 con 0x7fe1c40b4aa0 2024-04-26T21:09:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.277+0000 7fe1c9a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe1c40d2af0 con 0x7fe1c415ce40 2024-04-26T21:09:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.278+0000 7fe1c2000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe1c40d0b50 0x7fe1c40d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.278+0000 7fe1c2000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe1c40d0b50 0x7fe1c40d0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46700/0 (socket says 172.21.15.26:46700) 2024-04-26T21:09:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.278+0000 7fe1c2000640 1 -- 172.21.15.26:0/3730338995 learned_addr learned my addr 172.21.15.26:0/3730338995 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.278+0000 7fe1c3400640 1 --2- 172.21.15.26:0/3730338995 >> v2:172.21.15.86:3300/0 conn(0x7fe1c40b4aa0 0x7fe1c40b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.278+0000 7fe1c2a00640 1 --2- 172.21.15.26:0/3730338995 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe1c415ce40 0x7fe1c4162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.278+0000 7fe1c2000640 1 -- 172.21.15.26:0/3730338995 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe1c415ce40 msgr2=0x7fe1c4162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:13.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.278+0000 7fe1c2000640 1 --2- 172.21.15.26:0/3730338995 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe1c415ce40 0x7fe1c4162e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.278+0000 7fe1c2000640 1 -- 172.21.15.26:0/3730338995 >> v2:172.21.15.86:3300/0 conn(0x7fe1c40b4aa0 msgr2=0x7fe1c40b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:13.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.278+0000 7fe1c2000640 1 --2- 172.21.15.26:0/3730338995 >> v2:172.21.15.86:3300/0 conn(0x7fe1c40b4aa0 0x7fe1c40b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.278+0000 7fe1c2000640 1 -- 172.21.15.26:0/3730338995 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c40cfbb0 con 0x7fe1c40d0b50 2024-04-26T21:09:13.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.278+0000 7fe1c3400640 1 --2- 172.21.15.26:0/3730338995 >> v2:172.21.15.86:3300/0 conn(0x7fe1c40b4aa0 0x7fe1c40b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:09:13.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.280+0000 7fe1c2000640 1 --2- 172.21.15.26:0/3730338995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1c40d0b50 0x7fe1c40d0f90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40b2a50 tx=0x7fe1ac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3226a6fc46c8741 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:13.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.280+0000 7fe1c1600640 1 -- 172.21.15.26:0/3730338995 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac037070 con 0x7fe1c40d0b50 2024-04-26T21:09:13.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.280+0000 7fe1c1600640 1 -- 172.21.15.26:0/3730338995 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1ac032440 con 0x7fe1c40d0b50 2024-04-26T21:09:13.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.280+0000 7fe1c1600640 1 -- 172.21.15.26:0/3730338995 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac032810 con 0x7fe1c40d0b50 2024-04-26T21:09:13.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.281+0000 7fe1c9a00640 1 -- 172.21.15.26:0/3730338995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1c40d0b50 msgr2=0x7fe1c40d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:13.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.281+0000 7fe1c9a00640 1 --2- 172.21.15.26:0/3730338995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1c40d0b50 0x7fe1c40d0f90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40b2a50 tx=0x7fe1ac02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.281+0000 7fe1c9a00640 1 -- 172.21.15.26:0/3730338995 shutdown_connections 2024-04-26T21:09:13.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.281+0000 7fe1c9a00640 1 --2- 172.21.15.26:0/3730338995 >> v2:172.21.15.86:3300/0 conn(0x7fe1c40b4aa0 0x7fe1c40b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.281+0000 7fe1c9a00640 1 --2- 172.21.15.26:0/3730338995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1c40d0b50 0x7fe1c40d0f90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.281+0000 7fe1c9a00640 1 --2- 172.21.15.26:0/3730338995 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe1c415ce40 0x7fe1c4162e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.281+0000 7fe1c9a00640 1 -- 172.21.15.26:0/3730338995 >> 172.21.15.26:0/3730338995 conn(0x7fe1c40729a0 msgr2=0x7fe1c406c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:13.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.281+0000 7fe1c9a00640 1 -- 172.21.15.26:0/3730338995 shutdown_connections 2024-04-26T21:09:13.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.281+0000 7fe1c9a00640 1 -- 172.21.15.26:0/3730338995 wait complete. 2024-04-26T21:09:13.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.283+0000 7fe1c9a00640 1 Processor -- start 2024-04-26T21:09:13.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.283+0000 7fe1c9a00640 1 -- start start 2024-04-26T21:09:13.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.284+0000 7fe1c9a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1c40b4aa0 0x7fe1c40c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:13.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.284+0000 7fe1c9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1c40d0b50 0x7fe1c4148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:13.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.284+0000 7fe1c9a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe1c415ce40 0x7fe1c40d2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:13.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.284+0000 7fe1c9a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c40d0450 con 0x7fe1c40b4aa0 2024-04-26T21:09:13.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.284+0000 7fe1c9a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c40d0170 con 0x7fe1c40d0b50 2024-04-26T21:09:13.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.284+0000 7fe1c9a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe1c40d0730 con 0x7fe1c415ce40 2024-04-26T21:09:13.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.284+0000 7fe1c2a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1c40b4aa0 0x7fe1c40c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:13.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.284+0000 7fe1c2a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1c40b4aa0 0x7fe1c40c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46704/0 (socket says 172.21.15.26:46704) 2024-04-26T21:09:13.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.284+0000 7fe1c2a00640 1 -- 172.21.15.26:0/4084541600 learned_addr learned my addr 172.21.15.26:0/4084541600 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:13.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.284+0000 7fe1c2000640 1 --2- 172.21.15.26:0/4084541600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1c40d0b50 0x7fe1c4148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:13.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.284+0000 7fe1c2a00640 1 -- 172.21.15.26:0/4084541600 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe1c415ce40 msgr2=0x7fe1c40d2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:13.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.284+0000 7fe1c2a00640 1 --2- 172.21.15.26:0/4084541600 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe1c415ce40 0x7fe1c40d2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.284+0000 7fe1c2a00640 1 -- 172.21.15.26:0/4084541600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1c40d0b50 msgr2=0x7fe1c4148b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:13.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.284+0000 7fe1c2a00640 1 --2- 172.21.15.26:0/4084541600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1c40d0b50 0x7fe1c4148b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.284+0000 7fe1c2a00640 1 -- 172.21.15.26:0/4084541600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c414ef70 con 0x7fe1c40b4aa0 2024-04-26T21:09:13.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.284+0000 7fe1c2000640 1 --2- 172.21.15.26:0/4084541600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1c40d0b50 0x7fe1c4148b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:13.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.285+0000 7fe1c2a00640 1 --2- 172.21.15.26:0/4084541600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1c40b4aa0 0x7fe1c40c94e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4007e70 tx=0x7fe1b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:13.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.285+0000 7fe1baa00640 1 -- 172.21.15.26:0/4084541600 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4014620 con 0x7fe1c40b4aa0 2024-04-26T21:09:13.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.285+0000 7fe1baa00640 1 -- 172.21.15.26:0/4084541600 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1b4019070 con 0x7fe1c40b4aa0 2024-04-26T21:09:13.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.285+0000 7fe1baa00640 1 -- 172.21.15.26:0/4084541600 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4014a00 con 0x7fe1c40b4aa0 2024-04-26T21:09:13.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.285+0000 7fe1c9a00640 1 -- 172.21.15.26:0/4084541600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c40c9a20 con 0x7fe1c40b4aa0 2024-04-26T21:09:13.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.285+0000 7fe1c9a00640 1 -- 172.21.15.26:0/4084541600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c41f8a50 con 0x7fe1c40b4aa0 2024-04-26T21:09:13.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.286+0000 7fe1c9a00640 1 -- 172.21.15.26:0/4084541600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1c406a700 con 0x7fe1c40b4aa0 2024-04-26T21:09:13.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.287+0000 7fe1baa00640 1 -- 172.21.15.26:0/4084541600 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe1b4014ba0 con 0x7fe1c40b4aa0 2024-04-26T21:09:13.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.288+0000 7fe1baa00640 1 --2- 172.21.15.26:0/4084541600 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe1a006c8f0 0x7fe1a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:13.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.289+0000 7fe1baa00640 1 -- 172.21.15.26:0/4084541600 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe1b408c420 con 0x7fe1c40b4aa0 2024-04-26T21:09:13.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.289+0000 7fe1c2000640 1 --2- 172.21.15.26:0/4084541600 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe1a006c8f0 0x7fe1a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:13.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.290+0000 7fe1c2000640 1 --2- 172.21.15.26:0/4084541600 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe1a006c8f0 0x7fe1a006edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac034000 tx=0x7fe1ac02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:13.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.293+0000 7fe1baa00640 1 -- 172.21.15.26:0/4084541600 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe1b405a7b0 con 0x7fe1c40b4aa0 2024-04-26T21:09:13.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.491+0000 7fe1c9a00640 1 -- 172.21.15.26:0/4084541600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe1c4070e50 con 0x7fe1c40b4aa0 2024-04-26T21:09:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.492+0000 7fe1baa00640 1 -- 172.21.15.26:0/4084541600 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fe1b405a0b0 con 0x7fe1c40b4aa0 2024-04-26T21:09:13.494 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-26T21:09:13.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.495+0000 7fe1c9a00640 1 -- 172.21.15.26:0/4084541600 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe1a006c8f0 msgr2=0x7fe1a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:13.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.495+0000 7fe1c9a00640 1 --2- 172.21.15.26:0/4084541600 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe1a006c8f0 0x7fe1a006edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac034000 tx=0x7fe1ac02e000 comp rx=0 tx=0).stop 2024-04-26T21:09:13.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.495+0000 7fe1c9a00640 1 -- 172.21.15.26:0/4084541600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1c40b4aa0 msgr2=0x7fe1c40c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:13.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.495+0000 7fe1c9a00640 1 --2- 172.21.15.26:0/4084541600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1c40b4aa0 0x7fe1c40c94e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4007e70 tx=0x7fe1b400e4b0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.495+0000 7fe1c9a00640 1 -- 172.21.15.26:0/4084541600 shutdown_connections 2024-04-26T21:09:13.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.495+0000 7fe1c9a00640 1 --2- 172.21.15.26:0/4084541600 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe1a006c8f0 0x7fe1a006edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.495+0000 7fe1c9a00640 1 --2- 172.21.15.26:0/4084541600 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe1c415ce40 0x7fe1c40d2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.495+0000 7fe1c9a00640 1 --2- 172.21.15.26:0/4084541600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1c40d0b50 0x7fe1c4148b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.495+0000 7fe1c9a00640 1 --2- 172.21.15.26:0/4084541600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1c40b4aa0 0x7fe1c40c94e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.495+0000 7fe1c9a00640 1 -- 172.21.15.26:0/4084541600 >> 172.21.15.26:0/4084541600 conn(0x7fe1c40729a0 msgr2=0x7fe1c415aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:13.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.495+0000 7fe1c9a00640 1 -- 172.21.15.26:0/4084541600 shutdown_connections 2024-04-26T21:09:13.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.496+0000 7fe1c9a00640 1 -- 172.21.15.26:0/4084541600 wait complete. 2024-04-26T21:09:13.515 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:09:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.708+0000 7fd48e600640 1 Processor -- start 2024-04-26T21:09:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.708+0000 7fd48e600640 1 -- start start 2024-04-26T21:09:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.710+0000 7fd48e600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd4881469e0 0x7fd48815dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.710+0000 7fd48e600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd48806c240 0x7fd48806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.710+0000 7fd48e600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd48815e500 con 0x7fd488073160 2024-04-26T21:09:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.710+0000 7fd48e600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd4880d1500 con 0x7fd48806c240 2024-04-26T21:09:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.710+0000 7fd48e600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd4880d08f0 con 0x7fd4881469e0 2024-04-26T21:09:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.710+0000 7fd48dc00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd488073160 0x7fd4880735f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45364/0 (socket says 172.21.15.26:45364) 2024-04-26T21:09:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.710+0000 7fd48dc00640 1 -- 172.21.15.26:0/2273339659 learned_addr learned my addr 172.21.15.26:0/2273339659 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.711+0000 7fd48d200640 1 --2- 172.21.15.26:0/2273339659 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd4881469e0 0x7fd48815dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.711+0000 7fd487e00640 1 --2- 172.21.15.26:0/2273339659 >> v2:172.21.15.86:3300/0 conn(0x7fd48806c240 0x7fd48806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.711+0000 7fd487400640 1 -- 172.21.15.26:0/2273339659 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4128907679 0 0) 0x7fd480001490 con 0x7fd488073160 2024-04-26T21:09:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.711+0000 7fd487400640 1 -- 172.21.15.26:0/2273339659 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd470003520 con 0x7fd488073160 2024-04-26T21:09:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.711+0000 7fd487400640 1 -- 172.21.15.26:0/2273339659 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2804213374 0 0) 0x7fd470003520 con 0x7fd488073160 2024-04-26T21:09:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.711+0000 7fd487400640 1 -- 172.21.15.26:0/2273339659 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd4881469e0 msgr2=0x7fd48815dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.711+0000 7fd487400640 1 --2- 172.21.15.26:0/2273339659 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd4881469e0 0x7fd48815dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.711+0000 7fd487400640 1 -- 172.21.15.26:0/2273339659 >> v2:172.21.15.86:3300/0 conn(0x7fd48806c240 msgr2=0x7fd48806c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.711+0000 7fd487400640 1 --2- 172.21.15.26:0/2273339659 >> v2:172.21.15.86:3300/0 conn(0x7fd48806c240 0x7fd48806c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.711+0000 7fd487400640 1 -- 172.21.15.26:0/2273339659 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4880d0bd0 con 0x7fd488073160 2024-04-26T21:09:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.711+0000 7fd487400640 1 -- 172.21.15.26:0/2273339659 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7fd480001bc0 con 0x7fd488073160 2024-04-26T21:09:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.711+0000 7fd487400640 1 -- 172.21.15.26:0/2273339659 >> v1:172.21.15.26:6789/0 conn(0x7fd488073160 legacy=0x7fd4880735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.711+0000 7fd487400640 1 --2- 172.21.15.26:0/2273339659 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd4881469e0 0x7fd4700039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.711+0000 7fd487400640 1 --2- 172.21.15.26:0/2273339659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd470003f80 0x7fd470004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.711+0000 7fd487400640 1 --2- 172.21.15.26:0/2273339659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd470004d40 0x7fd4700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.711+0000 7fd487400640 1 -- 172.21.15.26:0/2273339659 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd4880d1500 con 0x7fd470003f80 2024-04-26T21:09:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.711+0000 7fd487400640 1 -- 172.21.15.26:0/2273339659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4880d08f0 con 0x7fd470004d40 2024-04-26T21:09:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.711+0000 7fd487400640 1 -- 172.21.15.26:0/2273339659 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd47000b9c0 con 0x7fd4881469e0 2024-04-26T21:09:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.712+0000 7fd487e00640 1 --2- 172.21.15.26:0/2273339659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd470003f80 0x7fd470004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.712+0000 7fd48dc00640 1 --2- 172.21.15.26:0/2273339659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd470004d40 0x7fd4700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.712+0000 7fd48d200640 1 --2- 172.21.15.26:0/2273339659 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd4881469e0 0x7fd4700039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.712+0000 7fd487e00640 1 -- 172.21.15.26:0/2273339659 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd4881469e0 msgr2=0x7fd4700039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:13.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.712+0000 7fd487e00640 1 --2- 172.21.15.26:0/2273339659 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd4881469e0 0x7fd4700039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.712+0000 7fd487e00640 1 -- 172.21.15.26:0/2273339659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd470004d40 msgr2=0x7fd4700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:13.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.712+0000 7fd487e00640 1 --2- 172.21.15.26:0/2273339659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd470004d40 0x7fd4700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.712+0000 7fd487e00640 1 -- 172.21.15.26:0/2273339659 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd470003520 con 0x7fd470003f80 2024-04-26T21:09:13.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.713+0000 7fd487e00640 1 --2- 172.21.15.26:0/2273339659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd470003f80 0x7fd470004800 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd474009af0 tx=0x7fd470005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446c70d4c516a7b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:13.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.714+0000 7fd487400640 1 -- 172.21.15.26:0/2273339659 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd474036070 con 0x7fd470003f80 2024-04-26T21:09:13.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.714+0000 7fd487400640 1 -- 172.21.15.26:0/2273339659 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd474031400 con 0x7fd470003f80 2024-04-26T21:09:13.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.714+0000 7fd487400640 1 -- 172.21.15.26:0/2273339659 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd474009690 con 0x7fd470003f80 2024-04-26T21:09:13.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.714+0000 7fd48e600640 1 -- 172.21.15.26:0/2273339659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd470003f80 msgr2=0x7fd470004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:13.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.714+0000 7fd48e600640 1 --2- 172.21.15.26:0/2273339659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd470003f80 0x7fd470004800 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd474009af0 tx=0x7fd470005830 comp rx=0 tx=0).stop 2024-04-26T21:09:13.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.714+0000 7fd48dc00640 1 -- 172.21.15.26:0/2273339659 reap_dead start 2024-04-26T21:09:13.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.715+0000 7fd48e600640 1 -- 172.21.15.26:0/2273339659 shutdown_connections 2024-04-26T21:09:13.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.715+0000 7fd48e600640 1 -- 172.21.15.26:0/2273339659 >> 172.21.15.26:0/2273339659 conn(0x7fd4880b3b00 msgr2=0x7fd4880b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:13.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.715+0000 7fd48e600640 1 -- 172.21.15.26:0/2273339659 shutdown_connections 2024-04-26T21:09:13.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.715+0000 7fd48e600640 1 -- 172.21.15.26:0/2273339659 wait complete. 2024-04-26T21:09:13.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.717+0000 7fd48e600640 1 Processor -- start 2024-04-26T21:09:13.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.717+0000 7fd48e600640 1 -- start start 2024-04-26T21:09:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.718+0000 7fd48e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd470003f80 0x7fd4881e5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.718+0000 7fd48e600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4881e6500 0x7fd48815bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.718+0000 7fd48e600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd4881f46e0 0x7fd48815c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.718+0000 7fd48e600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd4880716c0 con 0x7fd4881e6500 2024-04-26T21:09:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.718+0000 7fd48e600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4880d1190 con 0x7fd470003f80 2024-04-26T21:09:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.718+0000 7fd48e600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd488071840 con 0x7fd4881f46e0 2024-04-26T21:09:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.718+0000 7fd487e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4881e6500 0x7fd48815bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.718+0000 7fd487e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4881e6500 0x7fd48815bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46720/0 (socket says 172.21.15.26:46720) 2024-04-26T21:09:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.718+0000 7fd487e00640 1 -- 172.21.15.26:0/447116226 learned_addr learned my addr 172.21.15.26:0/447116226 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.718+0000 7fd48d200640 1 --2- 172.21.15.26:0/447116226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd470003f80 0x7fd4881e5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.719+0000 7fd487e00640 1 -- 172.21.15.26:0/447116226 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd4881f46e0 msgr2=0x7fd48815c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.719+0000 7fd487e00640 1 --2- 172.21.15.26:0/447116226 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd4881f46e0 0x7fd48815c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.719+0000 7fd487e00640 1 -- 172.21.15.26:0/447116226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd470003f80 msgr2=0x7fd4881e5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.719+0000 7fd487e00640 1 --2- 172.21.15.26:0/447116226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd470003f80 0x7fd4881e5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.719+0000 7fd487e00640 1 -- 172.21.15.26:0/447116226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4880b6440 con 0x7fd4881e6500 2024-04-26T21:09:13.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.719+0000 7fd487e00640 1 --2- 172.21.15.26:0/447116226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4881e6500 0x7fd48815bfd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd474034040 tx=0x7fd474004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:13.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.719+0000 7fd484c00640 1 -- 172.21.15.26:0/447116226 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd474040020 con 0x7fd4881e6500 2024-04-26T21:09:13.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.719+0000 7fd484c00640 1 -- 172.21.15.26:0/447116226 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd474003830 con 0x7fd4881e6500 2024-04-26T21:09:13.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.719+0000 7fd48e600640 1 -- 172.21.15.26:0/447116226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4880b6720 con 0x7fd4881e6500 2024-04-26T21:09:13.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.719+0000 7fd484c00640 1 -- 172.21.15.26:0/447116226 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd474036040 con 0x7fd4881e6500 2024-04-26T21:09:13.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.720+0000 7fd48e600640 1 -- 172.21.15.26:0/447116226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4880b6ce0 con 0x7fd4881e6500 2024-04-26T21:09:13.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.721+0000 7fd48e600640 1 -- 172.21.15.26:0/447116226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4880693d0 con 0x7fd4881e6500 2024-04-26T21:09:13.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.722+0000 7fd484c00640 1 -- 172.21.15.26:0/447116226 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd474043020 con 0x7fd4881e6500 2024-04-26T21:09:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.723+0000 7fd484c00640 1 --2- 172.21.15.26:0/447116226 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd46806c8f0 0x7fd46806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.723+0000 7fd484c00640 1 -- 172.21.15.26:0/447116226 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd4740abaf0 con 0x7fd4881e6500 2024-04-26T21:09:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.723+0000 7fd48d200640 1 --2- 172.21.15.26:0/447116226 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd46806c8f0 0x7fd46806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:13.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.724+0000 7fd48d200640 1 --2- 172.21.15.26:0/447116226 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd46806c8f0 0x7fd46806edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd4881e5af0 tx=0x7fd47c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:13.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.727+0000 7fd484c00640 1 -- 172.21.15.26:0/447116226 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd47404a070 con 0x7fd4881e6500 2024-04-26T21:09:13.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.933+0000 7fd48e600640 1 -- 172.21.15.26:0/447116226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd4880b3140 con 0x7fd4881e6500 2024-04-26T21:09:13.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.934+0000 7fd484c00640 1 -- 172.21.15.26:0/447116226 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fd474079e80 con 0x7fd4881e6500 2024-04-26T21:09:13.936 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-26T21:09:13.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.936+0000 7fd48e600640 1 -- 172.21.15.26:0/447116226 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd46806c8f0 msgr2=0x7fd46806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:13.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.936+0000 7fd48e600640 1 --2- 172.21.15.26:0/447116226 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd46806c8f0 0x7fd46806edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd4881e5af0 tx=0x7fd47c008040 comp rx=0 tx=0).stop 2024-04-26T21:09:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.937+0000 7fd48e600640 1 -- 172.21.15.26:0/447116226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4881e6500 msgr2=0x7fd48815bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.937+0000 7fd48e600640 1 --2- 172.21.15.26:0/447116226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4881e6500 0x7fd48815bfd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd474034040 tx=0x7fd474004730 comp rx=0 tx=0).stop 2024-04-26T21:09:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.937+0000 7fd48e600640 1 -- 172.21.15.26:0/447116226 shutdown_connections 2024-04-26T21:09:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.937+0000 7fd48e600640 1 --2- 172.21.15.26:0/447116226 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd46806c8f0 0x7fd46806edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.937+0000 7fd48e600640 1 --2- 172.21.15.26:0/447116226 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd4881f46e0 0x7fd48815c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.937+0000 7fd48e600640 1 --2- 172.21.15.26:0/447116226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4881e6500 0x7fd48815bfd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.937+0000 7fd48e600640 1 --2- 172.21.15.26:0/447116226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd470003f80 0x7fd4881e5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.937+0000 7fd48e600640 1 -- 172.21.15.26:0/447116226 >> 172.21.15.26:0/447116226 conn(0x7fd4880b3b00 msgr2=0x7fd48815e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.937+0000 7fd48e600640 1 -- 172.21.15.26:0/447116226 shutdown_connections 2024-04-26T21:09:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:13.937+0000 7fd48e600640 1 -- 172.21.15.26:0/447116226 wait complete. 2024-04-26T21:09:13.959 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:09:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.150+0000 7f6781c00640 1 Processor -- start 2024-04-26T21:09:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.151+0000 7f6781c00640 1 -- start start 2024-04-26T21:09:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.152+0000 7f6781c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f677c159730 0x7f677c160100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.152+0000 7f6781c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f677c160640 con 0x7f677c075540 2024-04-26T21:09:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.152+0000 7f6781c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f677c06e830 con 0x7f677c0b4b90 2024-04-26T21:09:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.152+0000 7f6781c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f677c06f8b0 con 0x7f677c159730 2024-04-26T21:09:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.152+0000 7f677b400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f677c075540 0x7f677c0b4480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45376/0 (socket says 172.21.15.26:45376) 2024-04-26T21:09:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.153+0000 7f677b400640 1 -- 172.21.15.26:0/4031895263 learned_addr learned my addr 172.21.15.26:0/4031895263 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.153+0000 7f677aa00640 1 -- 172.21.15.26:0/4031895263 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 591614720 0 0) 0x7f677c160640 con 0x7f677c075540 2024-04-26T21:09:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.153+0000 7f677be00640 1 --2- 172.21.15.26:0/4031895263 >> v2:172.21.15.86:3300/0 conn(0x7f677c159730 0x7f677c160100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.153+0000 7f677aa00640 1 -- 172.21.15.26:0/4031895263 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6760003520 con 0x7f677c075540 2024-04-26T21:09:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.153+0000 7f677aa00640 1 -- 172.21.15.26:0/4031895263 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1522431198 0 0) 0x7f6760003520 con 0x7f677c075540 2024-04-26T21:09:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.153+0000 7f677aa00640 1 -- 172.21.15.26:0/4031895263 >> v2:172.21.15.86:3300/0 conn(0x7f677c159730 msgr2=0x7f677c160100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.153+0000 7f677aa00640 1 --2- 172.21.15.26:0/4031895263 >> v2:172.21.15.86:3300/0 conn(0x7f677c159730 0x7f677c160100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.153+0000 7f677aa00640 1 -- 172.21.15.26:0/4031895263 >> v1:172.21.15.86:6789/0 conn(0x7f677c0b4b90 legacy=0x7f677c06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.153+0000 7f677aa00640 1 -- 172.21.15.26:0/4031895263 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677c06eaf0 con 0x7f677c075540 2024-04-26T21:09:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.154+0000 7f677aa00640 1 -- 172.21.15.26:0/4031895263 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f6764001670 con 0x7f677c075540 2024-04-26T21:09:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.154+0000 7f677aa00640 1 -- 172.21.15.26:0/4031895263 >> v1:172.21.15.26:6789/0 conn(0x7f677c075540 legacy=0x7f677c0b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.154+0000 7f677aa00640 1 --2- 172.21.15.26:0/4031895263 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6760003a00 0x7f6760003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.154+0000 7f677aa00640 1 --2- 172.21.15.26:0/4031895263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6760004800 0x7f6760004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.154+0000 7f677aa00640 1 --2- 172.21.15.26:0/4031895263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67600051c0 0x7f676000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.154+0000 7f677aa00640 1 -- 172.21.15.26:0/4031895263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f677c06f8b0 con 0x7f67600051c0 2024-04-26T21:09:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.154+0000 7f677aa00640 1 -- 172.21.15.26:0/4031895263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f676000c0c0 con 0x7f6760004800 2024-04-26T21:09:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.154+0000 7f677aa00640 1 -- 172.21.15.26:0/4031895263 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f676000c2c0 con 0x7f6760003a00 2024-04-26T21:09:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.154+0000 7f6781200640 1 --2- 172.21.15.26:0/4031895263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67600051c0 0x7f676000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.154+0000 7f677b400640 1 --2- 172.21.15.26:0/4031895263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6760004800 0x7f6760004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.154+0000 7f677be00640 1 --2- 172.21.15.26:0/4031895263 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6760003a00 0x7f6760003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.154+0000 7f6781200640 1 -- 172.21.15.26:0/4031895263 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6760003a00 msgr2=0x7f6760003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.154+0000 7f6781200640 1 --2- 172.21.15.26:0/4031895263 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6760003a00 0x7f6760003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.154+0000 7f6781200640 1 -- 172.21.15.26:0/4031895263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6760004800 msgr2=0x7f6760004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.154+0000 7f6781200640 1 --2- 172.21.15.26:0/4031895263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6760004800 0x7f6760004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.154+0000 7f6781200640 1 -- 172.21.15.26:0/4031895263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6760003520 con 0x7f67600051c0 2024-04-26T21:09:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.154+0000 7f677b400640 1 --2- 172.21.15.26:0/4031895263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6760004800 0x7f6760004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.155+0000 7f6781200640 1 -- 172.21.15.26:0/4031895263 reap_dead start 2024-04-26T21:09:14.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.155+0000 7f6781200640 1 --2- 172.21.15.26:0/4031895263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67600051c0 0x7f676000b9c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6774001750 tx=0x7f6760005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd19ce355e22354 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:14.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.156+0000 7f677aa00640 1 -- 172.21.15.26:0/4031895263 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6774001370 con 0x7f67600051c0 2024-04-26T21:09:14.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.156+0000 7f677aa00640 1 -- 172.21.15.26:0/4031895263 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6774005310 con 0x7f67600051c0 2024-04-26T21:09:14.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.156+0000 7f677aa00640 1 -- 172.21.15.26:0/4031895263 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677402abe0 con 0x7f67600051c0 2024-04-26T21:09:14.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.156+0000 7f6781c00640 1 -- 172.21.15.26:0/4031895263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67600051c0 msgr2=0x7f676000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:14.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.156+0000 7f6781c00640 1 --2- 172.21.15.26:0/4031895263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67600051c0 0x7f676000b9c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6774001750 tx=0x7f6760005830 comp rx=0 tx=0).stop 2024-04-26T21:09:14.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.156+0000 7f6781c00640 1 -- 172.21.15.26:0/4031895263 shutdown_connections 2024-04-26T21:09:14.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.156+0000 7f6781c00640 1 --2- 172.21.15.26:0/4031895263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67600051c0 0x7f676000b9c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:14.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.156+0000 7f6781c00640 1 -- 172.21.15.26:0/4031895263 >> 172.21.15.26:0/4031895263 conn(0x7f677c0d1470 msgr2=0x7f677c0d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:14.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.156+0000 7f6781c00640 1 -- 172.21.15.26:0/4031895263 shutdown_connections 2024-04-26T21:09:14.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.157+0000 7f6781c00640 1 -- 172.21.15.26:0/4031895263 wait complete. 2024-04-26T21:09:14.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.158+0000 7f6781c00640 1 Processor -- start 2024-04-26T21:09:14.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.158+0000 7f6781c00640 1 -- start start 2024-04-26T21:09:14.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.159+0000 7f6781c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67600051c0 0x7f677c152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:14.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.159+0000 7f6781c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f677c13d7e0 0x7f677c162fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.159+0000 7f6781c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f677c163500 0x7f677c1400c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.159+0000 7f6781c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f677c06f390 con 0x7f677c163500 2024-04-26T21:09:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.159+0000 7f6781c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f677c06f0b0 con 0x7f67600051c0 2024-04-26T21:09:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.159+0000 7f6781c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f677c159b10 con 0x7f677c13d7e0 2024-04-26T21:09:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.159+0000 7f6781200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f677c163500 0x7f677c1400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.160+0000 7f6781200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f677c163500 0x7f677c1400c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46736/0 (socket says 172.21.15.26:46736) 2024-04-26T21:09:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.160+0000 7f6781200640 1 -- 172.21.15.26:0/2027406433 learned_addr learned my addr 172.21.15.26:0/2027406433 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.160+0000 7f677b400640 1 --2- 172.21.15.26:0/2027406433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f677c13d7e0 0x7f677c162fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.160+0000 7f6781200640 1 -- 172.21.15.26:0/2027406433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f677c13d7e0 msgr2=0x7f677c162fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.160+0000 7f677be00640 1 --2- 172.21.15.26:0/2027406433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67600051c0 0x7f677c152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.160+0000 7f6781200640 1 --2- 172.21.15.26:0/2027406433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f677c13d7e0 0x7f677c162fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.160+0000 7f6781200640 1 -- 172.21.15.26:0/2027406433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67600051c0 msgr2=0x7f677c152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.160+0000 7f6781200640 1 --2- 172.21.15.26:0/2027406433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67600051c0 0x7f677c152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.160+0000 7f6781200640 1 -- 172.21.15.26:0/2027406433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677c152f70 con 0x7f677c163500 2024-04-26T21:09:14.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.160+0000 7f677b400640 1 --2- 172.21.15.26:0/2027406433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f677c13d7e0 0x7f677c162fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:14.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.160+0000 7f677be00640 1 --2- 172.21.15.26:0/2027406433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67600051c0 0x7f677c152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:14.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.160+0000 7f6781200640 1 --2- 172.21.15.26:0/2027406433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f677c163500 0x7f677c1400c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6774001870 tx=0x7f6774001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:14.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.161+0000 7f6773e00640 1 -- 172.21.15.26:0/2027406433 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6774001370 con 0x7f677c163500 2024-04-26T21:09:14.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.161+0000 7f6773e00640 1 -- 172.21.15.26:0/2027406433 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6774000ed0 con 0x7f677c163500 2024-04-26T21:09:14.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.161+0000 7f6773e00640 1 -- 172.21.15.26:0/2027406433 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6774007a30 con 0x7f677c163500 2024-04-26T21:09:14.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.161+0000 7f6781c00640 1 -- 172.21.15.26:0/2027406433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677c13e160 con 0x7f677c163500 2024-04-26T21:09:14.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.161+0000 7f6781c00640 1 -- 172.21.15.26:0/2027406433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677c13e720 con 0x7f677c163500 2024-04-26T21:09:14.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.162+0000 7f6781c00640 1 -- 172.21.15.26:0/2027406433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f677c0506c0 con 0x7f677c163500 2024-04-26T21:09:14.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.163+0000 7f6773e00640 1 -- 172.21.15.26:0/2027406433 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6774007030 con 0x7f677c163500 2024-04-26T21:09:14.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.164+0000 7f6773e00640 1 --2- 172.21.15.26:0/2027406433 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f675806c8f0 0x7f675806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:14.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.164+0000 7f6773e00640 1 -- 172.21.15.26:0/2027406433 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f67740aad10 con 0x7f677c163500 2024-04-26T21:09:14.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.164+0000 7f677be00640 1 --2- 172.21.15.26:0/2027406433 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f675806c8f0 0x7f675806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:14.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.165+0000 7f677be00640 1 --2- 172.21.15.26:0/2027406433 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f675806c8f0 0x7f675806edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f676c007460 tx=0x7f676c00d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:14.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.168+0000 7f6773e00640 1 -- 172.21.15.26:0/2027406433 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f677c152f70 con 0x7f677c163500 2024-04-26T21:09:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.374+0000 7f6781c00640 1 -- 172.21.15.26:0/2027406433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f677c1574b0 con 0x7f677c163500 2024-04-26T21:09:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.375+0000 7f6773e00640 1 -- 172.21.15.26:0/2027406433 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f677402ae20 con 0x7f677c163500 2024-04-26T21:09:14.377 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-26T21:09:14.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.378+0000 7f6781c00640 1 -- 172.21.15.26:0/2027406433 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f675806c8f0 msgr2=0x7f675806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:14.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.378+0000 7f6781c00640 1 --2- 172.21.15.26:0/2027406433 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f675806c8f0 0x7f675806edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f676c007460 tx=0x7f676c00d040 comp rx=0 tx=0).stop 2024-04-26T21:09:14.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.378+0000 7f6781c00640 1 -- 172.21.15.26:0/2027406433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f677c163500 msgr2=0x7f677c1400c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:14.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.378+0000 7f6781c00640 1 --2- 172.21.15.26:0/2027406433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f677c163500 0x7f677c1400c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6774001870 tx=0x7f6774001120 comp rx=0 tx=0).stop 2024-04-26T21:09:14.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.378+0000 7f6781c00640 1 -- 172.21.15.26:0/2027406433 shutdown_connections 2024-04-26T21:09:14.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.378+0000 7f6781c00640 1 --2- 172.21.15.26:0/2027406433 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f675806c8f0 0x7f675806edb0 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f676c007460 tx=0x7f676c00d040 comp rx=0 tx=0).stop 2024-04-26T21:09:14.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.378+0000 7f6781c00640 1 --2- 172.21.15.26:0/2027406433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f677c163500 0x7f677c1400c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:14.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.378+0000 7f6781c00640 1 --2- 172.21.15.26:0/2027406433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f677c13d7e0 0x7f677c162fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:14.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.378+0000 7f6781c00640 1 --2- 172.21.15.26:0/2027406433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67600051c0 0x7f677c152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:14.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.378+0000 7f6781c00640 1 -- 172.21.15.26:0/2027406433 >> 172.21.15.26:0/2027406433 conn(0x7f677c0d1470 msgr2=0x7f677c0b2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:14.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.378+0000 7f6781c00640 1 -- 172.21.15.26:0/2027406433 shutdown_connections 2024-04-26T21:09:14.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.378+0000 7f6781c00640 1 -- 172.21.15.26:0/2027406433 wait complete. 2024-04-26T21:09:14.399 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:09:14.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.600+0000 7f14bf400640 1 Processor -- start 2024-04-26T21:09:14.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.600+0000 7f14bf400640 1 -- start start 2024-04-26T21:09:14.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.601+0000 7f14bf400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f14b80cf1c0 0x7f14b806f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:14.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.601+0000 7f14bf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14b80b6a40 0x7f14b806f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:14.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.602+0000 7f14bf400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14b815bbc0 con 0x7f14b8072ac0 2024-04-26T21:09:14.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.602+0000 7f14bf400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f14b806eec0 con 0x7f14b80cf1c0 2024-04-26T21:09:14.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.602+0000 7f14bf400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f14b80d09b0 con 0x7f14b80b6a40 2024-04-26T21:09:14.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.602+0000 7f14bca00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f14b80cf1c0 0x7f14b806f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:14.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.602+0000 7f14bd400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f14b8072ac0 0x7f14b80b4030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45392/0 (socket says 172.21.15.26:45392) 2024-04-26T21:09:14.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.602+0000 7f14bd400640 1 -- 172.21.15.26:0/3251534476 learned_addr learned my addr 172.21.15.26:0/3251534476 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:14.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.602+0000 7f14b7e00640 1 --2- 172.21.15.26:0/3251534476 >> v2:172.21.15.86:3300/0 conn(0x7f14b80b6a40 0x7f14b806f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:14.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.602+0000 7f14bca00640 1 -- 172.21.15.26:0/3251534476 >> v2:172.21.15.86:3300/0 conn(0x7f14b80b6a40 msgr2=0x7f14b806f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:14.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.602+0000 7f14bca00640 1 --2- 172.21.15.26:0/3251534476 >> v2:172.21.15.86:3300/0 conn(0x7f14b80b6a40 0x7f14b806f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:14.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.602+0000 7f14bca00640 1 -- 172.21.15.26:0/3251534476 >> v1:172.21.15.26:6789/0 conn(0x7f14b8072ac0 legacy=0x7f14b80b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:14.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.602+0000 7f14bca00640 1 -- 172.21.15.26:0/3251534476 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b80d0240 con 0x7f14b80cf1c0 2024-04-26T21:09:14.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.602+0000 7f14b7400640 1 -- 172.21.15.26:0/3251534476 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2001458141 0 0) 0x7f14b815bbc0 con 0x7f14b8072ac0 2024-04-26T21:09:14.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.604+0000 7f14bca00640 1 --2- 172.21.15.26:0/3251534476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14b80cf1c0 0x7f14b806f350 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f14b80ced10 tx=0x7f14ac02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d81381e87143102 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:14.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.604+0000 7f14b7400640 1 -- 172.21.15.26:0/3251534476 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac037070 con 0x7f14b80cf1c0 2024-04-26T21:09:14.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.604+0000 7f14b7400640 1 -- 172.21.15.26:0/3251534476 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14ac032440 con 0x7f14b80cf1c0 2024-04-26T21:09:14.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.604+0000 7f14b7400640 1 -- 172.21.15.26:0/3251534476 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac02ae60 con 0x7f14b80cf1c0 2024-04-26T21:09:14.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.604+0000 7f14bf400640 1 -- 172.21.15.26:0/3251534476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14b80cf1c0 msgr2=0x7f14b806f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:14.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.604+0000 7f14bf400640 1 --2- 172.21.15.26:0/3251534476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14b80cf1c0 0x7f14b806f350 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f14b80ced10 tx=0x7f14ac02a110 comp rx=0 tx=0).stop 2024-04-26T21:09:14.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.604+0000 7f14bf400640 1 -- 172.21.15.26:0/3251534476 shutdown_connections 2024-04-26T21:09:14.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.604+0000 7f14bf400640 1 --2- 172.21.15.26:0/3251534476 >> v2:172.21.15.86:3300/0 conn(0x7f14b80b6a40 0x7f14b806f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:14.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.604+0000 7f14bf400640 1 --2- 172.21.15.26:0/3251534476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14b80cf1c0 0x7f14b806f350 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:14.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.604+0000 7f14bf400640 1 -- 172.21.15.26:0/3251534476 >> 172.21.15.26:0/3251534476 conn(0x7f14b80cd840 msgr2=0x7f14b80cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:14.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.605+0000 7f14bf400640 1 -- 172.21.15.26:0/3251534476 shutdown_connections 2024-04-26T21:09:14.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.605+0000 7f14bf400640 1 -- 172.21.15.26:0/3251534476 wait complete. 2024-04-26T21:09:14.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.606+0000 7f14bf400640 1 Processor -- start 2024-04-26T21:09:14.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.607+0000 7f14bf400640 1 -- start start 2024-04-26T21:09:14.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.608+0000 7f14bf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14b8072ac0 0x7f14b80cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:14.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.608+0000 7f14bf400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14b80b6a40 0x7f14b81e6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:14.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.608+0000 7f14bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14b80cf1c0 0x7f14b81e72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:14.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.608+0000 7f14bf400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f14b80b48c0 con 0x7f14b8072ac0 2024-04-26T21:09:14.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.608+0000 7f14bf400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14b80b4740 con 0x7f14b80cf1c0 2024-04-26T21:09:14.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.608+0000 7f14bf400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f14b80d0d80 con 0x7f14b80b6a40 2024-04-26T21:09:14.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.608+0000 7f14bca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14b8072ac0 0x7f14b80cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:14.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.608+0000 7f14bca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14b8072ac0 0x7f14b80cbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46752/0 (socket says 172.21.15.26:46752) 2024-04-26T21:09:14.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.608+0000 7f14bca00640 1 -- 172.21.15.26:0/2741377516 learned_addr learned my addr 172.21.15.26:0/2741377516 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:14.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.608+0000 7f14b7e00640 1 --2- 172.21.15.26:0/2741377516 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14b80b6a40 0x7f14b81e6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:14.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.608+0000 7f14bca00640 1 -- 172.21.15.26:0/2741377516 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14b80b6a40 msgr2=0x7f14b81e6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:14.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.608+0000 7f14bca00640 1 --2- 172.21.15.26:0/2741377516 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14b80b6a40 0x7f14b81e6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:14.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.608+0000 7f14bd400640 1 --2- 172.21.15.26:0/2741377516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14b80cf1c0 0x7f14b81e72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:14.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.608+0000 7f14bca00640 1 -- 172.21.15.26:0/2741377516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14b80cf1c0 msgr2=0x7f14b81e72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:14.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.608+0000 7f14bca00640 1 --2- 172.21.15.26:0/2741377516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14b80cf1c0 0x7f14b81e72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:14.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.608+0000 7f14bca00640 1 -- 172.21.15.26:0/2741377516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b80cc880 con 0x7f14b8072ac0 2024-04-26T21:09:14.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.609+0000 7f14bca00640 1 --2- 172.21.15.26:0/2741377516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14b8072ac0 0x7f14b80cbfe0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009890 tx=0x7f14ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:14.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.609+0000 7f14b4c00640 1 -- 172.21.15.26:0/2741377516 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac040020 con 0x7f14b8072ac0 2024-04-26T21:09:14.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.609+0000 7f14b4c00640 1 -- 172.21.15.26:0/2741377516 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14ac038040 con 0x7f14b8072ac0 2024-04-26T21:09:14.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.609+0000 7f14b4c00640 1 -- 172.21.15.26:0/2741377516 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac037070 con 0x7f14b8072ac0 2024-04-26T21:09:14.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.609+0000 7f14bf400640 1 -- 172.21.15.26:0/2741377516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b80ccb60 con 0x7f14b8072ac0 2024-04-26T21:09:14.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.609+0000 7f14bf400640 1 -- 172.21.15.26:0/2741377516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b80cd120 con 0x7f14b8072ac0 2024-04-26T21:09:14.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.611+0000 7f14bf400640 1 -- 172.21.15.26:0/2741377516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14b806a700 con 0x7f14b8072ac0 2024-04-26T21:09:14.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.612+0000 7f14b4c00640 1 -- 172.21.15.26:0/2741377516 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f14ac043020 con 0x7f14b8072ac0 2024-04-26T21:09:14.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.612+0000 7f14b4c00640 1 --2- 172.21.15.26:0/2741377516 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f149806c8f0 0x7f149806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:14.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.613+0000 7f14b4c00640 1 -- 172.21.15.26:0/2741377516 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f14ac0abf30 con 0x7f14b8072ac0 2024-04-26T21:09:14.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.613+0000 7f14b7e00640 1 --2- 172.21.15.26:0/2741377516 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f149806c8f0 0x7f149806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:14.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.613+0000 7f14b7e00640 1 --2- 172.21.15.26:0/2741377516 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f149806c8f0 0x7f149806edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f14b8157130 tx=0x7f14a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:14.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.617+0000 7f14b4c00640 1 -- 172.21.15.26:0/2741377516 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f14ac04a070 con 0x7f14b8072ac0 2024-04-26T21:09:14.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.819+0000 7f14bf400640 1 -- 172.21.15.26:0/2741377516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f14b806ead0 con 0x7f14b8072ac0 2024-04-26T21:09:14.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.822+0000 7f14b4c00640 1 -- 172.21.15.26:0/2741377516 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16557 (secure 0 0 0) 0x7f14ac032e20 con 0x7f14b8072ac0 2024-04-26T21:09:14.825 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:14.825 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:08:39.177854+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:08:35.481169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:08:36.609498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:09:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.825+0000 7f14bf400640 1 -- 172.21.15.26:0/2741377516 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f149806c8f0 msgr2=0x7f149806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.825+0000 7f14bf400640 1 --2- 172.21.15.26:0/2741377516 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f149806c8f0 0x7f149806edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f14b8157130 tx=0x7f14a4008040 comp rx=0 tx=0).stop 2024-04-26T21:09:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.825+0000 7f14bf400640 1 -- 172.21.15.26:0/2741377516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14b8072ac0 msgr2=0x7f14b80cbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.825+0000 7f14bf400640 1 --2- 172.21.15.26:0/2741377516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14b8072ac0 0x7f14b80cbfe0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009890 tx=0x7f14ac0046c0 comp rx=0 tx=0).stop 2024-04-26T21:09:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.825+0000 7f14bf400640 1 -- 172.21.15.26:0/2741377516 shutdown_connections 2024-04-26T21:09:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.825+0000 7f14bf400640 1 --2- 172.21.15.26:0/2741377516 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f149806c8f0 0x7f149806edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.825+0000 7f14bf400640 1 --2- 172.21.15.26:0/2741377516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14b80cf1c0 0x7f14b81e72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.825+0000 7f14bf400640 1 --2- 172.21.15.26:0/2741377516 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14b80b6a40 0x7f14b81e6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.825+0000 7f14bf400640 1 --2- 172.21.15.26:0/2741377516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14b8072ac0 0x7f14b80cbfe0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.825+0000 7f14bf400640 1 -- 172.21.15.26:0/2741377516 >> 172.21.15.26:0/2741377516 conn(0x7f14b80cd840 msgr2=0x7f14b815e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.826+0000 7f14bf400640 1 -- 172.21.15.26:0/2741377516 shutdown_connections 2024-04-26T21:09:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:14.826+0000 7f14bf400640 1 -- 172.21.15.26:0/2741377516 wait complete. 2024-04-26T21:09:14.845 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:08:31.322511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:08:35.481169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:08:36.609498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:09:14.846 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:09:15.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.066+0000 7f1efcc00640 1 Processor -- start 2024-04-26T21:09:15.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.066+0000 7f1efcc00640 1 -- start start 2024-04-26T21:09:15.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.068+0000 7f1efcc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ef815bdc0 0x7f1ef80b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:15.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.068+0000 7f1efcc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1ef806e210 0x7f1ef8072e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:15.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.068+0000 7f1efcc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ef80726b0 0x7f1ef8073360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:15.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.068+0000 7f1efcc00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f1ef80b4cf0 con 0x7f1ef80726b0 2024-04-26T21:09:15.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.068+0000 7f1efcc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1ef80cf280 con 0x7f1ef806e210 2024-04-26T21:09:15.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.068+0000 7f1efcc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef806fbb0 con 0x7f1ef815bdc0 2024-04-26T21:09:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.069+0000 7f1ef7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ef80726b0 0x7f1ef8073360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.069+0000 7f1ef7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ef80726b0 0x7f1ef8073360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46754/0 (socket says 172.21.15.26:46754) 2024-04-26T21:09:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.069+0000 7f1ef7e00640 1 -- 172.21.15.26:0/1530078289 learned_addr learned my addr 172.21.15.26:0/1530078289 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.069+0000 7f1eefe00640 1 --2- 172.21.15.26:0/1530078289 >> v2:172.21.15.86:3300/0 conn(0x7f1ef806e210 0x7f1ef8072e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.069+0000 7f1ef7400640 1 --2- 172.21.15.26:0/1530078289 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ef815bdc0 0x7f1ef80b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.069+0000 7f1ef7e00640 1 -- 172.21.15.26:0/1530078289 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ef815bdc0 msgr2=0x7f1ef80b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.069+0000 7f1ef7e00640 1 --2- 172.21.15.26:0/1530078289 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ef815bdc0 0x7f1ef80b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.069+0000 7f1ef7e00640 1 -- 172.21.15.26:0/1530078289 >> v2:172.21.15.86:3300/0 conn(0x7f1ef806e210 msgr2=0x7f1ef8072e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.069+0000 7f1ef7e00640 1 --2- 172.21.15.26:0/1530078289 >> v2:172.21.15.86:3300/0 conn(0x7f1ef806e210 0x7f1ef8072e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.069+0000 7f1ef7e00640 1 -- 172.21.15.26:0/1530078289 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef806e780 con 0x7f1ef80726b0 2024-04-26T21:09:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.070+0000 7f1ef7e00640 1 --2- 172.21.15.26:0/1530078289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ef80726b0 0x7f1ef8073360 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8074bd0 tx=0x7f1ef002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a3c081d4d897f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.071+0000 7f1ef6a00640 1 -- 172.21.15.26:0/1530078289 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0036070 con 0x7f1ef80726b0 2024-04-26T21:09:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.071+0000 7f1ef6a00640 1 -- 172.21.15.26:0/1530078289 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ef0031600 con 0x7f1ef80726b0 2024-04-26T21:09:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.071+0000 7f1ef6a00640 1 -- 172.21.15.26:0/1530078289 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef00319d0 con 0x7f1ef80726b0 2024-04-26T21:09:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.072+0000 7f1efcc00640 1 -- 172.21.15.26:0/1530078289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ef80726b0 msgr2=0x7f1ef8073360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.072+0000 7f1efcc00640 1 --2- 172.21.15.26:0/1530078289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ef80726b0 0x7f1ef8073360 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8074bd0 tx=0x7f1ef002a610 comp rx=0 tx=0).stop 2024-04-26T21:09:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.072+0000 7f1efcc00640 1 -- 172.21.15.26:0/1530078289 shutdown_connections 2024-04-26T21:09:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.072+0000 7f1efcc00640 1 --2- 172.21.15.26:0/1530078289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ef80726b0 0x7f1ef8073360 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.072+0000 7f1efcc00640 1 --2- 172.21.15.26:0/1530078289 >> v2:172.21.15.86:3300/0 conn(0x7f1ef806e210 0x7f1ef8072e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.072+0000 7f1efcc00640 1 --2- 172.21.15.26:0/1530078289 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ef815bdc0 0x7f1ef80b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.072+0000 7f1efcc00640 1 -- 172.21.15.26:0/1530078289 >> 172.21.15.26:0/1530078289 conn(0x7f1ef80b2a10 msgr2=0x7f1ef80b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.072+0000 7f1efcc00640 1 -- 172.21.15.26:0/1530078289 shutdown_connections 2024-04-26T21:09:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.072+0000 7f1efcc00640 1 -- 172.21.15.26:0/1530078289 wait complete. 2024-04-26T21:09:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.074+0000 7f1efcc00640 1 Processor -- start 2024-04-26T21:09:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.074+0000 7f1efcc00640 1 -- start start 2024-04-26T21:09:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.075+0000 7f1efcc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ef806e210 0x7f1ef8143070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.076+0000 7f1efcc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ef80726b0 0x7f1ef815f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.076+0000 7f1efcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef815bdc0 0x7f1ef813fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.076+0000 7f1efcc00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef8072380 con 0x7f1ef80726b0 2024-04-26T21:09:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.076+0000 7f1efcc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef8070130 con 0x7f1ef815bdc0 2024-04-26T21:09:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.076+0000 7f1efcc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef815e2d0 con 0x7f1ef806e210 2024-04-26T21:09:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.076+0000 7f1eefe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ef80726b0 0x7f1ef815f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.076+0000 7f1eefe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ef80726b0 0x7f1ef815f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46768/0 (socket says 172.21.15.26:46768) 2024-04-26T21:09:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.076+0000 7f1eefe00640 1 -- 172.21.15.26:0/4272062661 learned_addr learned my addr 172.21.15.26:0/4272062661 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.076+0000 7f1ef7400640 1 --2- 172.21.15.26:0/4272062661 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ef806e210 0x7f1ef8143070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.076+0000 7f1ef7e00640 1 --2- 172.21.15.26:0/4272062661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef815bdc0 0x7f1ef813fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.076+0000 7f1eefe00640 1 -- 172.21.15.26:0/4272062661 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ef806e210 msgr2=0x7f1ef8143070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.076+0000 7f1eefe00640 1 --2- 172.21.15.26:0/4272062661 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ef806e210 0x7f1ef8143070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.076+0000 7f1eefe00640 1 -- 172.21.15.26:0/4272062661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef815bdc0 msgr2=0x7f1ef813fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.076+0000 7f1eefe00640 1 --2- 172.21.15.26:0/4272062661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef815bdc0 0x7f1ef813fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.076+0000 7f1eefe00640 1 -- 172.21.15.26:0/4272062661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef815e5b0 con 0x7f1ef80726b0 2024-04-26T21:09:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.077+0000 7f1ef7e00640 1 --2- 172.21.15.26:0/4272062661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef815bdc0 0x7f1ef813fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.077+0000 7f1eefe00640 1 --2- 172.21.15.26:0/4272062661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ef80726b0 0x7f1ef815f370 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1ee000e890 tx=0x7f1ee000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.077+0000 7f1eef400640 1 -- 172.21.15.26:0/4272062661 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0017070 con 0x7f1ef80726b0 2024-04-26T21:09:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.077+0000 7f1eef400640 1 -- 172.21.15.26:0/4272062661 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee000c940 con 0x7f1ef80726b0 2024-04-26T21:09:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.077+0000 7f1efcc00640 1 -- 172.21.15.26:0/4272062661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef815e890 con 0x7f1ef80726b0 2024-04-26T21:09:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.077+0000 7f1eef400640 1 -- 172.21.15.26:0/4272062661 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee000cd10 con 0x7f1ef80726b0 2024-04-26T21:09:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.077+0000 7f1efcc00640 1 -- 172.21.15.26:0/4272062661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef815ee50 con 0x7f1ef80726b0 2024-04-26T21:09:15.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.078+0000 7f1efcc00640 1 -- 172.21.15.26:0/4272062661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ef806a700 con 0x7f1ef80726b0 2024-04-26T21:09:15.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.080+0000 7f1eef400640 1 -- 172.21.15.26:0/4272062661 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ee001a070 con 0x7f1ef80726b0 2024-04-26T21:09:15.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.081+0000 7f1eef400640 1 --2- 172.21.15.26:0/4272062661 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1ed406c8f0 0x7f1ed406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:15.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.081+0000 7f1eef400640 1 -- 172.21.15.26:0/4272062661 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1ee008bf30 con 0x7f1ef80726b0 2024-04-26T21:09:15.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.082+0000 7f1ef7400640 1 --2- 172.21.15.26:0/4272062661 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1ed406c8f0 0x7f1ed406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:15.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.082+0000 7f1ef7400640 1 --2- 172.21.15.26:0/4272062661 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1ed406c8f0 0x7f1ed406edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009a30 tx=0x7f1ee8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:15.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.086+0000 7f1eef400640 1 -- 172.21.15.26:0/4272062661 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1ee005a2c0 con 0x7f1ef80726b0 2024-04-26T21:09:15.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.287+0000 7f1efcc00640 1 -- 172.21.15.26:0/4272062661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1ef8158ae0 con 0x7f1ef80726b0 2024-04-26T21:09:15.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.288+0000 7f1eef400640 1 -- 172.21.15.26:0/4272062661 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f1ee0059bc0 con 0x7f1ef80726b0 2024-04-26T21:09:15.290 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-26T21:09:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.291+0000 7f1efcc00640 1 -- 172.21.15.26:0/4272062661 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1ed406c8f0 msgr2=0x7f1ed406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.291+0000 7f1efcc00640 1 --2- 172.21.15.26:0/4272062661 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1ed406c8f0 0x7f1ed406edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009a30 tx=0x7f1ee8008040 comp rx=0 tx=0).stop 2024-04-26T21:09:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.291+0000 7f1efcc00640 1 -- 172.21.15.26:0/4272062661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ef80726b0 msgr2=0x7f1ef815f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.291+0000 7f1efcc00640 1 --2- 172.21.15.26:0/4272062661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ef80726b0 0x7f1ef815f370 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1ee000e890 tx=0x7f1ee000ed50 comp rx=0 tx=0).stop 2024-04-26T21:09:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.291+0000 7f1efcc00640 1 -- 172.21.15.26:0/4272062661 shutdown_connections 2024-04-26T21:09:15.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.291+0000 7f1efcc00640 1 --2- 172.21.15.26:0/4272062661 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1ed406c8f0 0x7f1ed406edb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.291+0000 7f1efcc00640 1 --2- 172.21.15.26:0/4272062661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef815bdc0 0x7f1ef813fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.291+0000 7f1efcc00640 1 --2- 172.21.15.26:0/4272062661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ef80726b0 0x7f1ef815f370 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.291+0000 7f1efcc00640 1 --2- 172.21.15.26:0/4272062661 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ef806e210 0x7f1ef8143070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.291+0000 7f1efcc00640 1 -- 172.21.15.26:0/4272062661 >> 172.21.15.26:0/4272062661 conn(0x7f1ef80b2a10 msgr2=0x7f1ef80b5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.291+0000 7f1efcc00640 1 -- 172.21.15.26:0/4272062661 shutdown_connections 2024-04-26T21:09:15.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.292+0000 7f1efcc00640 1 -- 172.21.15.26:0/4272062661 wait complete. 2024-04-26T21:09:15.312 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:09:15.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.530+0000 7f28e9a00640 1 Processor -- start 2024-04-26T21:09:15.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.530+0000 7f28e9a00640 1 -- start start 2024-04-26T21:09:15.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.532+0000 7f28e9a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28e40b5d30 0x7f28e40b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.532+0000 7f28e9a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28e40b42c0 0x7f28e40b6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.532+0000 7f28e9a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28e4146c00 con 0x7f28e40b6650 2024-04-26T21:09:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.532+0000 7f28e9a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f28e4073700 con 0x7f28e40b42c0 2024-04-26T21:09:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.532+0000 7f28e9a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f28e406e420 con 0x7f28e40b5d30 2024-04-26T21:09:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.532+0000 7f28e2000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28e40b42c0 0x7f28e40b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.532+0000 7f28e2000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28e40b42c0 0x7f28e40b6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46772/0 (socket says 172.21.15.26:46772) 2024-04-26T21:09:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.532+0000 7f28e2000640 1 -- 172.21.15.26:0/3848747100 learned_addr learned my addr 172.21.15.26:0/3848747100 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.532+0000 7f28e2a00640 1 --2- 172.21.15.26:0/3848747100 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28e40b5d30 0x7f28e40b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.533+0000 7f28e2000640 1 -- 172.21.15.26:0/3848747100 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28e40b5d30 msgr2=0x7f28e40b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.533+0000 7f28e2000640 1 --2- 172.21.15.26:0/3848747100 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28e40b5d30 0x7f28e40b3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.533+0000 7f28e2000640 1 -- 172.21.15.26:0/3848747100 >> v1:172.21.15.86:6789/0 conn(0x7f28e40b6650 legacy=0x7f28e406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.533+0000 7f28e2000640 1 -- 172.21.15.26:0/3848747100 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e4072e60 con 0x7f28e40b42c0 2024-04-26T21:09:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.534+0000 7f28e2000640 1 --2- 172.21.15.26:0/3848747100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28e40b42c0 0x7f28e40b6110 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009870 tx=0x7f28d002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621d59dce1797c1f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:15.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.534+0000 7f28e1600640 1 -- 172.21.15.26:0/3848747100 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0037070 con 0x7f28e40b42c0 2024-04-26T21:09:15.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.534+0000 7f28e1600640 1 -- 172.21.15.26:0/3848747100 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28d0032440 con 0x7f28e40b42c0 2024-04-26T21:09:15.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.534+0000 7f28e1600640 1 -- 172.21.15.26:0/3848747100 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0032810 con 0x7f28e40b42c0 2024-04-26T21:09:15.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.535+0000 7f28e9a00640 1 -- 172.21.15.26:0/3848747100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28e40b42c0 msgr2=0x7f28e40b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:15.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.535+0000 7f28e9a00640 1 --2- 172.21.15.26:0/3848747100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28e40b42c0 0x7f28e40b6110 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009870 tx=0x7f28d002a210 comp rx=0 tx=0).stop 2024-04-26T21:09:15.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.535+0000 7f28e9a00640 1 -- 172.21.15.26:0/3848747100 shutdown_connections 2024-04-26T21:09:15.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.535+0000 7f28e9a00640 1 --2- 172.21.15.26:0/3848747100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28e40b42c0 0x7f28e40b6110 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:15.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.535+0000 7f28e9a00640 1 --2- 172.21.15.26:0/3848747100 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28e40b5d30 0x7f28e40b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:15.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.535+0000 7f28e9a00640 1 -- 172.21.15.26:0/3848747100 >> 172.21.15.26:0/3848747100 conn(0x7f28e40b2a80 msgr2=0x7f28e40b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:15.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.535+0000 7f28e9a00640 1 -- 172.21.15.26:0/3848747100 shutdown_connections 2024-04-26T21:09:15.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.535+0000 7f28e9a00640 1 -- 172.21.15.26:0/3848747100 wait complete. 2024-04-26T21:09:15.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.537+0000 7f28e9a00640 1 Processor -- start 2024-04-26T21:09:15.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.537+0000 7f28e9a00640 1 -- start start 2024-04-26T21:09:15.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.538+0000 7f28e9a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28e40b42c0 0x7f28e41e3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:15.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.538+0000 7f28e9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28e40b5d30 0x7f28e40afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:15.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.538+0000 7f28e9a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28e40b6650 0x7f28e40ab130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:15.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.538+0000 7f28e9a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f28e4072a80 con 0x7f28e40b6650 2024-04-26T21:09:15.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.538+0000 7f28e9a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28e4073420 con 0x7f28e40b5d30 2024-04-26T21:09:15.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.538+0000 7f28e9a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f28e4072c00 con 0x7f28e40b42c0 2024-04-26T21:09:15.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.539+0000 7f28e3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28e40b6650 0x7f28e40ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:15.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.539+0000 7f28e3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28e40b6650 0x7f28e40ab130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46782/0 (socket says 172.21.15.26:46782) 2024-04-26T21:09:15.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.539+0000 7f28e3400640 1 -- 172.21.15.26:0/2118807328 learned_addr learned my addr 172.21.15.26:0/2118807328 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:15.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.539+0000 7f28e2a00640 1 --2- 172.21.15.26:0/2118807328 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28e40b42c0 0x7f28e41e3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:15.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.539+0000 7f28e2000640 1 --2- 172.21.15.26:0/2118807328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28e40b5d30 0x7f28e40afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:15.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.539+0000 7f28e3400640 1 -- 172.21.15.26:0/2118807328 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28e40b42c0 msgr2=0x7f28e41e3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:15.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.539+0000 7f28e3400640 1 --2- 172.21.15.26:0/2118807328 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28e40b42c0 0x7f28e41e3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:15.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.539+0000 7f28e3400640 1 -- 172.21.15.26:0/2118807328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28e40b5d30 msgr2=0x7f28e40afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:15.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.539+0000 7f28e3400640 1 --2- 172.21.15.26:0/2118807328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28e40b5d30 0x7f28e40afba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:15.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.539+0000 7f28e3400640 1 -- 172.21.15.26:0/2118807328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e40aed50 con 0x7f28e40b6650 2024-04-26T21:09:15.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.540+0000 7f28e3400640 1 --2- 172.21.15.26:0/2118807328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28e40b6650 0x7f28e40ab130 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00be60 tx=0x7f28dc00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:15.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.540+0000 7f28daa00640 1 -- 172.21.15.26:0/2118807328 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc019070 con 0x7f28e40b6650 2024-04-26T21:09:15.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.540+0000 7f28daa00640 1 -- 172.21.15.26:0/2118807328 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28dc014440 con 0x7f28e40b6650 2024-04-26T21:09:15.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.540+0000 7f28daa00640 1 -- 172.21.15.26:0/2118807328 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc014870 con 0x7f28e40b6650 2024-04-26T21:09:15.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.540+0000 7f28e9a00640 1 -- 172.21.15.26:0/2118807328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e40aeff0 con 0x7f28e40b6650 2024-04-26T21:09:15.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.540+0000 7f28e9a00640 1 -- 172.21.15.26:0/2118807328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e40af5b0 con 0x7f28e40b6650 2024-04-26T21:09:15.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.541+0000 7f28e9a00640 1 -- 172.21.15.26:0/2118807328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28e406a700 con 0x7f28e40b6650 2024-04-26T21:09:15.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.542+0000 7f28daa00640 1 -- 172.21.15.26:0/2118807328 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28dc01c070 con 0x7f28e40b6650 2024-04-26T21:09:15.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.543+0000 7f28daa00640 1 --2- 172.21.15.26:0/2118807328 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f28c406c940 0x7f28c406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:15.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.543+0000 7f28daa00640 1 -- 172.21.15.26:0/2118807328 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f28dc08bb00 con 0x7f28e40b6650 2024-04-26T21:09:15.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.543+0000 7f28e2a00640 1 --2- 172.21.15.26:0/2118807328 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f28c406c940 0x7f28c406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:15.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.544+0000 7f28e2a00640 1 --2- 172.21.15.26:0/2118807328 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f28c406c940 0x7f28c406ee00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f28d4006fd0 tx=0x7f28d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:15.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.548+0000 7f28daa00640 1 -- 172.21.15.26:0/2118807328 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f28dc059f10 con 0x7f28e40b6650 2024-04-26T21:09:15.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.754+0000 7f28e9a00640 1 -- 172.21.15.26:0/2118807328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f28e4072780 con 0x7f28e40b6650 2024-04-26T21:09:15.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.755+0000 7f28daa00640 1 -- 172.21.15.26:0/2118807328 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f28dc059810 con 0x7f28e40b6650 2024-04-26T21:09:15.757 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-26T21:09:15.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.757+0000 7f28e9a00640 1 -- 172.21.15.26:0/2118807328 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f28c406c940 msgr2=0x7f28c406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:15.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.757+0000 7f28e9a00640 1 --2- 172.21.15.26:0/2118807328 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f28c406c940 0x7f28c406ee00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f28d4006fd0 tx=0x7f28d4008040 comp rx=0 tx=0).stop 2024-04-26T21:09:15.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.757+0000 7f28e9a00640 1 -- 172.21.15.26:0/2118807328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28e40b6650 msgr2=0x7f28e40ab130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:15.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.757+0000 7f28e9a00640 1 --2- 172.21.15.26:0/2118807328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28e40b6650 0x7f28e40ab130 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00be60 tx=0x7f28dc00bf50 comp rx=0 tx=0).stop 2024-04-26T21:09:15.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.758+0000 7f28e9a00640 1 -- 172.21.15.26:0/2118807328 shutdown_connections 2024-04-26T21:09:15.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.758+0000 7f28e9a00640 1 --2- 172.21.15.26:0/2118807328 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f28c406c940 0x7f28c406ee00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:15.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.758+0000 7f28e9a00640 1 --2- 172.21.15.26:0/2118807328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28e40b6650 0x7f28e40ab130 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:15.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.758+0000 7f28e9a00640 1 --2- 172.21.15.26:0/2118807328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28e40b5d30 0x7f28e40afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:15.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.758+0000 7f28e9a00640 1 --2- 172.21.15.26:0/2118807328 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28e40b42c0 0x7f28e41e3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:15.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.758+0000 7f28e9a00640 1 -- 172.21.15.26:0/2118807328 >> 172.21.15.26:0/2118807328 conn(0x7f28e40b2a80 msgr2=0x7f28e406f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:15.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.758+0000 7f28e9a00640 1 -- 172.21.15.26:0/2118807328 shutdown_connections 2024-04-26T21:09:15.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.758+0000 7f28e9a00640 1 -- 172.21.15.26:0/2118807328 wait complete. 2024-04-26T21:09:15.778 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:09:15.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.997+0000 7f55c8400640 1 Processor -- start 2024-04-26T21:09:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.997+0000 7f55c8400640 1 -- start start 2024-04-26T21:09:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.998+0000 7f55c8400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f55c015f2b0 0x7f55c00d5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.998+0000 7f55c8400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55c015f690 0x7f55c00d1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.998+0000 7f55c8400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55c00d3d10 con 0x7f55c00d3770 2024-04-26T21:09:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.998+0000 7f55c8400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f55c00d3ff0 con 0x7f55c015f2b0 2024-04-26T21:09:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.998+0000 7f55c8400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f55c00b8190 con 0x7f55c015f690 2024-04-26T21:09:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.999+0000 7f55c5000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55c015f690 0x7f55c00d1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:16.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.999+0000 7f55c5000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55c015f690 0x7f55c00d1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.26:53266/0 (socket says 172.21.15.26:53266) 2024-04-26T21:09:16.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.999+0000 7f55c5000640 1 -- 172.21.15.26:0/3213176862 learned_addr learned my addr 172.21.15.26:0/3213176862 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:16.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:15.999+0000 7f55c5a00640 1 --2- 172.21.15.26:0/3213176862 >> v2:172.21.15.86:3300/0 conn(0x7f55c015f2b0 0x7f55c00d5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:16.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.000+0000 7f55c5000640 1 -- 172.21.15.26:0/3213176862 >> v2:172.21.15.86:3300/0 conn(0x7f55c015f2b0 msgr2=0x7f55c00d5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:16.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.000+0000 7f55c5000640 1 --2- 172.21.15.26:0/3213176862 >> v2:172.21.15.86:3300/0 conn(0x7f55c015f2b0 0x7f55c00d5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.000+0000 7f55c5000640 1 -- 172.21.15.26:0/3213176862 >> v1:172.21.15.86:6789/0 conn(0x7f55c00d3770 legacy=0x7f55c00cfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:16.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.000+0000 7f55c5000640 1 -- 172.21.15.26:0/3213176862 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c00b8470 con 0x7f55c015f690 2024-04-26T21:09:16.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.001+0000 7f55c5000640 1 --2- 172.21.15.26:0/3213176862 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55c015f690 0x7f55c00d1a90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009e50 tx=0x7f55ac02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ca36c13e78de5ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:16.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.001+0000 7f55bfe00640 1 -- 172.21.15.26:0/3213176862 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac036070 con 0x7f55c015f690 2024-04-26T21:09:16.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.001+0000 7f55bfe00640 1 -- 172.21.15.26:0/3213176862 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55ac031600 con 0x7f55c015f690 2024-04-26T21:09:16.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.001+0000 7f55bfe00640 1 -- 172.21.15.26:0/3213176862 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac0319d0 con 0x7f55c015f690 2024-04-26T21:09:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.002+0000 7f55c8400640 1 -- 172.21.15.26:0/3213176862 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55c015f690 msgr2=0x7f55c00d1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.002+0000 7f55c8400640 1 --2- 172.21.15.26:0/3213176862 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55c015f690 0x7f55c00d1a90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009e50 tx=0x7f55ac02a680 comp rx=0 tx=0).stop 2024-04-26T21:09:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.002+0000 7f55c8400640 1 -- 172.21.15.26:0/3213176862 shutdown_connections 2024-04-26T21:09:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.002+0000 7f55c8400640 1 --2- 172.21.15.26:0/3213176862 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55c015f690 0x7f55c00d1a90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.002+0000 7f55c8400640 1 --2- 172.21.15.26:0/3213176862 >> v2:172.21.15.86:3300/0 conn(0x7f55c015f2b0 0x7f55c00d5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.002+0000 7f55c8400640 1 -- 172.21.15.26:0/3213176862 >> 172.21.15.26:0/3213176862 conn(0x7f55c009e490 msgr2=0x7f55c00d51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.002+0000 7f55c8400640 1 -- 172.21.15.26:0/3213176862 shutdown_connections 2024-04-26T21:09:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.002+0000 7f55c8400640 1 -- 172.21.15.26:0/3213176862 wait complete. 2024-04-26T21:09:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.004+0000 7f55c8400640 1 Processor -- start 2024-04-26T21:09:16.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.004+0000 7f55c8400640 1 -- start start 2024-04-26T21:09:16.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.005+0000 7f55c8400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c00d3770 0x7f55c00c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:16.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.005+0000 7f55c8400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55c015f2b0 0x7f55c0159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:16.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.005+0000 7f55c8400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55c015f690 0x7f55c00ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:16.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.005+0000 7f55c8400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f55c00b9b20 con 0x7f55c00d3770 2024-04-26T21:09:16.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.005+0000 7f55c8400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55c00b99a0 con 0x7f55c015f2b0 2024-04-26T21:09:16.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.005+0000 7f55c8400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f55c00b9ca0 con 0x7f55c015f690 2024-04-26T21:09:16.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.006+0000 7f55c5a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c00d3770 0x7f55c00c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:16.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.006+0000 7f55c5a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c00d3770 0x7f55c00c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46790/0 (socket says 172.21.15.26:46790) 2024-04-26T21:09:16.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.006+0000 7f55c5a00640 1 -- 172.21.15.26:0/77939140 learned_addr learned my addr 172.21.15.26:0/77939140 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:16.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.006+0000 7f55c5000640 1 --2- 172.21.15.26:0/77939140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55c015f2b0 0x7f55c0159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:16.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.006+0000 7f55c6400640 1 --2- 172.21.15.26:0/77939140 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55c015f690 0x7f55c00ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:16.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.006+0000 7f55c5a00640 1 -- 172.21.15.26:0/77939140 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55c015f690 msgr2=0x7f55c00ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:16.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.006+0000 7f55c5a00640 1 --2- 172.21.15.26:0/77939140 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55c015f690 0x7f55c00ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.006+0000 7f55c5a00640 1 -- 172.21.15.26:0/77939140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55c015f2b0 msgr2=0x7f55c0159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:16.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.006+0000 7f55c5a00640 1 --2- 172.21.15.26:0/77939140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55c015f2b0 0x7f55c0159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.006+0000 7f55c5a00640 1 -- 172.21.15.26:0/77939140 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c00cd480 con 0x7f55c00d3770 2024-04-26T21:09:16.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.006+0000 7f55c5000640 1 --2- 172.21.15.26:0/77939140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55c015f2b0 0x7f55c0159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:16.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.006+0000 7f55c6400640 1 --2- 172.21.15.26:0/77939140 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55c015f690 0x7f55c00ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:16.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.006+0000 7f55c5a00640 1 --2- 172.21.15.26:0/77939140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c00d3770 0x7f55c00c9ef0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f55b000e8a0 tx=0x7f55b000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.007+0000 7f55bd600640 1 -- 172.21.15.26:0/77939140 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0017070 con 0x7f55c00d3770 2024-04-26T21:09:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.007+0000 7f55bd600640 1 -- 172.21.15.26:0/77939140 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b000c940 con 0x7f55c00d3770 2024-04-26T21:09:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.007+0000 7f55bd600640 1 -- 172.21.15.26:0/77939140 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b000cd30 con 0x7f55c00d3770 2024-04-26T21:09:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.007+0000 7f55c8400640 1 -- 172.21.15.26:0/77939140 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c00cd760 con 0x7f55c00d3770 2024-04-26T21:09:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.007+0000 7f55c8400640 1 -- 172.21.15.26:0/77939140 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c00cdd20 con 0x7f55c00d3770 2024-04-26T21:09:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.008+0000 7f55c8400640 1 -- 172.21.15.26:0/77939140 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55c0157c80 con 0x7f55c00d3770 2024-04-26T21:09:16.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.009+0000 7f55bd600640 1 -- 172.21.15.26:0/77939140 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f55b001a070 con 0x7f55c00d3770 2024-04-26T21:09:16.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.010+0000 7f55bd600640 1 --2- 172.21.15.26:0/77939140 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f55a006c970 0x7f55a006ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:16.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.010+0000 7f55bd600640 1 -- 172.21.15.26:0/77939140 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f55b008bd60 con 0x7f55c00d3770 2024-04-26T21:09:16.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.010+0000 7f55c5000640 1 --2- 172.21.15.26:0/77939140 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f55a006c970 0x7f55a006ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:16.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.011+0000 7f55c5000640 1 --2- 172.21.15.26:0/77939140 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f55a006c970 0x7f55a006ee30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f55ac033000 tx=0x7f55ac009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:16.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.015+0000 7f55bd600640 1 -- 172.21.15.26:0/77939140 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f55c0157c80 con 0x7f55c00d3770 2024-04-26T21:09:16.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.224+0000 7f55c8400640 1 -- 172.21.15.26:0/77939140 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f55c0050730 con 0x7f55c00d3770 2024-04-26T21:09:16.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.225+0000 7f55bd600640 1 -- 172.21.15.26:0/77939140 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f55b005a170 con 0x7f55c00d3770 2024-04-26T21:09:16.227 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-26T21:09:16.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.228+0000 7f55c8400640 1 -- 172.21.15.26:0/77939140 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f55a006c970 msgr2=0x7f55a006ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:16.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.228+0000 7f55c8400640 1 --2- 172.21.15.26:0/77939140 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f55a006c970 0x7f55a006ee30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f55ac033000 tx=0x7f55ac009000 comp rx=0 tx=0).stop 2024-04-26T21:09:16.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.228+0000 7f55c8400640 1 -- 172.21.15.26:0/77939140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c00d3770 msgr2=0x7f55c00c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:16.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.228+0000 7f55c8400640 1 --2- 172.21.15.26:0/77939140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c00d3770 0x7f55c00c9ef0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f55b000e8a0 tx=0x7f55b000ed60 comp rx=0 tx=0).stop 2024-04-26T21:09:16.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.228+0000 7f55c8400640 1 -- 172.21.15.26:0/77939140 shutdown_connections 2024-04-26T21:09:16.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.228+0000 7f55c8400640 1 --2- 172.21.15.26:0/77939140 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f55a006c970 0x7f55a006ee30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.228+0000 7f55c8400640 1 --2- 172.21.15.26:0/77939140 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55c015f690 0x7f55c00ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.228+0000 7f55c8400640 1 --2- 172.21.15.26:0/77939140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55c015f2b0 0x7f55c0159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.228+0000 7f55c8400640 1 --2- 172.21.15.26:0/77939140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c00d3770 0x7f55c00c9ef0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.228+0000 7f55c8400640 1 -- 172.21.15.26:0/77939140 >> 172.21.15.26:0/77939140 conn(0x7f55c009e490 msgr2=0x7f55c006cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:16.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.228+0000 7f55c8400640 1 -- 172.21.15.26:0/77939140 shutdown_connections 2024-04-26T21:09:16.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.228+0000 7f55c8400640 1 -- 172.21.15.26:0/77939140 wait complete. 2024-04-26T21:09:16.249 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:09:16.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.460+0000 7f1ff0e00640 1 Processor -- start 2024-04-26T21:09:16.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.460+0000 7f1ff0e00640 1 -- start start 2024-04-26T21:09:16.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.462+0000 7f1ff0e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fec15dcb0 0x7f1fec164590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:16.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.462+0000 7f1ff0e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1fec06ebb0 0x7f1fec0d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:16.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.462+0000 7f1ff0e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fec164ad0 con 0x7f1fec06e700 2024-04-26T21:09:16.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.462+0000 7f1ff0e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f1fec15e090 con 0x7f1fec06ebb0 2024-04-26T21:09:16.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.462+0000 7f1ff0e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1fec072500 con 0x7f1fec15dcb0 2024-04-26T21:09:16.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.462+0000 7f1febe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1fec06ebb0 0x7f1fec0d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:16.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.462+0000 7f1feaa00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1fec06e700 0x7f1fec1677a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45394/0 (socket says 172.21.15.26:45394) 2024-04-26T21:09:16.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.463+0000 7f1febe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1fec06ebb0 0x7f1fec0d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46798/0 (socket says 172.21.15.26:46798) 2024-04-26T21:09:16.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.463+0000 7f1feaa00640 1 -- 172.21.15.26:0/3403857419 learned_addr learned my addr 172.21.15.26:0/3403857419 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:16.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.463+0000 7f1feb400640 1 --2- 172.21.15.26:0/3403857419 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fec15dcb0 0x7f1fec164590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:16.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.463+0000 7f1febe00640 1 -- 172.21.15.26:0/3403857419 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fec15dcb0 msgr2=0x7f1fec164590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:16.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.463+0000 7f1febe00640 1 --2- 172.21.15.26:0/3403857419 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fec15dcb0 0x7f1fec164590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.463+0000 7f1febe00640 1 -- 172.21.15.26:0/3403857419 >> v1:172.21.15.26:6789/0 conn(0x7f1fec06e700 legacy=0x7f1fec1677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:16.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.463+0000 7f1febe00640 1 -- 172.21.15.26:0/3403857419 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fec0727e0 con 0x7f1fec06ebb0 2024-04-26T21:09:16.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.464+0000 7f1febe00640 1 --2- 172.21.15.26:0/3403857419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fec06ebb0 0x7f1fec0d1130 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009d90 tx=0x7f1fe402a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7ff358ea86a981 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:16.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.464+0000 7f1fea000640 1 -- 172.21.15.26:0/3403857419 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4007740 con 0x7f1fec06ebb0 2024-04-26T21:09:16.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.464+0000 7f1fea000640 1 -- 172.21.15.26:0/3403857419 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe40078e0 con 0x7f1fec06ebb0 2024-04-26T21:09:16.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.464+0000 7f1fea000640 1 -- 172.21.15.26:0/3403857419 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4007cb0 con 0x7f1fec06ebb0 2024-04-26T21:09:16.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.465+0000 7f1ff0e00640 1 -- 172.21.15.26:0/3403857419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fec06ebb0 msgr2=0x7f1fec0d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:16.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.465+0000 7f1ff0e00640 1 --2- 172.21.15.26:0/3403857419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fec06ebb0 0x7f1fec0d1130 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009d90 tx=0x7f1fe402a540 comp rx=0 tx=0).stop 2024-04-26T21:09:16.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.465+0000 7f1ff0e00640 1 -- 172.21.15.26:0/3403857419 shutdown_connections 2024-04-26T21:09:16.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.465+0000 7f1ff0e00640 1 --2- 172.21.15.26:0/3403857419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fec06ebb0 0x7f1fec0d1130 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.465+0000 7f1ff0e00640 1 --2- 172.21.15.26:0/3403857419 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fec15dcb0 0x7f1fec164590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.465+0000 7f1ff0e00640 1 -- 172.21.15.26:0/3403857419 >> 172.21.15.26:0/3403857419 conn(0x7f1fec06c690 msgr2=0x7f1fec06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:16.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.465+0000 7f1ff0e00640 1 -- 172.21.15.26:0/3403857419 shutdown_connections 2024-04-26T21:09:16.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.465+0000 7f1ff0e00640 1 -- 172.21.15.26:0/3403857419 wait complete. 2024-04-26T21:09:16.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.467+0000 7f1ff0e00640 1 Processor -- start 2024-04-26T21:09:16.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.467+0000 7f1ff0e00640 1 -- start start 2024-04-26T21:09:16.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.468+0000 7f1ff0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fec06e700 0x7f1fec1488a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:16.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.468+0000 7f1ff0e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fec06ebb0 0x7f1fec0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:16.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.468+0000 7f1ff0e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fec15dcb0 0x7f1fec0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:16.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.468+0000 7f1ff0e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec072da0 con 0x7f1fec06ebb0 2024-04-26T21:09:16.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.468+0000 7f1ff0e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec167280 con 0x7f1fec06e700 2024-04-26T21:09:16.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.468+0000 7f1ff0e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1fec06e3c0 con 0x7f1fec15dcb0 2024-04-26T21:09:16.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.469+0000 7f1feaa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fec06ebb0 0x7f1fec0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:16.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.469+0000 7f1feaa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fec06ebb0 0x7f1fec0b1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46802/0 (socket says 172.21.15.26:46802) 2024-04-26T21:09:16.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.469+0000 7f1feaa00640 1 -- 172.21.15.26:0/1647179944 learned_addr learned my addr 172.21.15.26:0/1647179944 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:16.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.469+0000 7f1febe00640 1 --2- 172.21.15.26:0/1647179944 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fec15dcb0 0x7f1fec0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:16.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.469+0000 7f1feb400640 1 --2- 172.21.15.26:0/1647179944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fec06e700 0x7f1fec1488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:16.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.469+0000 7f1feaa00640 1 -- 172.21.15.26:0/1647179944 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fec15dcb0 msgr2=0x7f1fec0b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:16.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.469+0000 7f1feaa00640 1 --2- 172.21.15.26:0/1647179944 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fec15dcb0 0x7f1fec0b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.469+0000 7f1feaa00640 1 -- 172.21.15.26:0/1647179944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fec06e700 msgr2=0x7f1fec1488a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:16.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.469+0000 7f1feaa00640 1 --2- 172.21.15.26:0/1647179944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fec06e700 0x7f1fec1488a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.469+0000 7f1feaa00640 1 -- 172.21.15.26:0/1647179944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fec156fb0 con 0x7f1fec06ebb0 2024-04-26T21:09:16.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.470+0000 7f1feaa00640 1 --2- 172.21.15.26:0/1647179944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fec06ebb0 0x7f1fec0b1ac0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400c3c0 tx=0x7f1fd400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:16.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.470+0000 7f1fe3400640 1 -- 172.21.15.26:0/1647179944 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4017070 con 0x7f1fec06ebb0 2024-04-26T21:09:16.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.470+0000 7f1fe3400640 1 -- 172.21.15.26:0/1647179944 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fd4005ce0 con 0x7f1fec06ebb0 2024-04-26T21:09:16.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.470+0000 7f1fe3400640 1 -- 172.21.15.26:0/1647179944 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd40072f0 con 0x7f1fec06ebb0 2024-04-26T21:09:16.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.470+0000 7f1ff0e00640 1 -- 172.21.15.26:0/1647179944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fec157290 con 0x7f1fec06ebb0 2024-04-26T21:09:16.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.470+0000 7f1ff0e00640 1 -- 172.21.15.26:0/1647179944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fec159520 con 0x7f1fec06ebb0 2024-04-26T21:09:16.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.471+0000 7f1ff0e00640 1 -- 172.21.15.26:0/1647179944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fec06a700 con 0x7f1fec06ebb0 2024-04-26T21:09:16.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.472+0000 7f1fe3400640 1 -- 172.21.15.26:0/1647179944 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1fd401a070 con 0x7f1fec06ebb0 2024-04-26T21:09:16.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.473+0000 7f1fe3400640 1 --2- 172.21.15.26:0/1647179944 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1fc806c940 0x7f1fc806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:16.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.473+0000 7f1fe3400640 1 -- 172.21.15.26:0/1647179944 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1fd408aea0 con 0x7f1fec06ebb0 2024-04-26T21:09:16.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.474+0000 7f1feb400640 1 --2- 172.21.15.26:0/1647179944 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1fc806c940 0x7f1fc806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:16.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.474+0000 7f1feb400640 1 --2- 172.21.15.26:0/1647179944 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1fc806c940 0x7f1fc806ee00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc006fd0 tx=0x7f1fdc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.478+0000 7f1fe3400640 1 -- 172.21.15.26:0/1647179944 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1fd4059340 con 0x7f1fec06ebb0 2024-04-26T21:09:16.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.683+0000 7f1ff0e00640 1 -- 172.21.15.26:0/1647179944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1fec0b5da0 con 0x7f1fec06ebb0 2024-04-26T21:09:16.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.686+0000 7f1fe3400640 1 -- 172.21.15.26:0/1647179944 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16557 (secure 0 0 0) 0x7f1fd4058c40 con 0x7f1fec06ebb0 2024-04-26T21:09:16.688 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:16.689 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:08:39.177854+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:08:35.481169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:08:36.609498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:09:16.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.689+0000 7f1ff0e00640 1 -- 172.21.15.26:0/1647179944 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1fc806c940 msgr2=0x7f1fc806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:16.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.689+0000 7f1ff0e00640 1 --2- 172.21.15.26:0/1647179944 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1fc806c940 0x7f1fc806ee00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc006fd0 tx=0x7f1fdc008040 comp rx=0 tx=0).stop 2024-04-26T21:09:16.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.689+0000 7f1ff0e00640 1 -- 172.21.15.26:0/1647179944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fec06ebb0 msgr2=0x7f1fec0b1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:16.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.689+0000 7f1ff0e00640 1 --2- 172.21.15.26:0/1647179944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fec06ebb0 0x7f1fec0b1ac0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400c3c0 tx=0x7f1fd400c880 comp rx=0 tx=0).stop 2024-04-26T21:09:16.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.689+0000 7f1ff0e00640 1 -- 172.21.15.26:0/1647179944 shutdown_connections 2024-04-26T21:09:16.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.689+0000 7f1ff0e00640 1 --2- 172.21.15.26:0/1647179944 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1fc806c940 0x7f1fc806ee00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.689+0000 7f1ff0e00640 1 --2- 172.21.15.26:0/1647179944 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fec15dcb0 0x7f1fec0b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.689+0000 7f1ff0e00640 1 --2- 172.21.15.26:0/1647179944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fec06ebb0 0x7f1fec0b1ac0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.689+0000 7f1ff0e00640 1 --2- 172.21.15.26:0/1647179944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fec06e700 0x7f1fec1488a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.689+0000 7f1ff0e00640 1 -- 172.21.15.26:0/1647179944 >> 172.21.15.26:0/1647179944 conn(0x7f1fec06c690 msgr2=0x7f1fec0b4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:16.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.690+0000 7f1ff0e00640 1 -- 172.21.15.26:0/1647179944 shutdown_connections 2024-04-26T21:09:16.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.690+0000 7f1ff0e00640 1 -- 172.21.15.26:0/1647179944 wait complete. 2024-04-26T21:09:16.710 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:08:31.322511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:08:35.481169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:08:36.609498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:09:16.711 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:09:16.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.921+0000 7f63cbe00640 1 Processor -- start 2024-04-26T21:09:16.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.921+0000 7f63cbe00640 1 -- start start 2024-04-26T21:09:16.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.922+0000 7f63cbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f63cc160a80 0x7f63cc0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:16.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.923+0000 7f63cbe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f63cc160e60 0x7f63cc06e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:16.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.923+0000 7f63cbe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63cc072d90 con 0x7f63cc072490 2024-04-26T21:09:16.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.923+0000 7f63cbe00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f63cc0b5be0 con 0x7f63cc160e60 2024-04-26T21:09:16.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.923+0000 7f63cbe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f63cc073070 con 0x7f63cc160a80 2024-04-26T21:09:16.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.923+0000 7f63ca000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f63cc160e60 0x7f63cc06e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:16.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.923+0000 7f63ca000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f63cc160e60 0x7f63cc06e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46814/0 (socket says 172.21.15.26:46814) 2024-04-26T21:09:16.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.923+0000 7f63ca000640 1 -- 172.21.15.26:0/2837282444 learned_addr learned my addr 172.21.15.26:0/2837282444 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:16.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.923+0000 7f63caa00640 1 --2- 172.21.15.26:0/2837282444 >> v2:172.21.15.86:3300/0 conn(0x7f63cc160a80 0x7f63cc0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:16.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.923+0000 7f63ca000640 1 -- 172.21.15.26:0/2837282444 >> v2:172.21.15.86:3300/0 conn(0x7f63cc160a80 msgr2=0x7f63cc0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:16.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.924+0000 7f63ca000640 1 --2- 172.21.15.26:0/2837282444 >> v2:172.21.15.86:3300/0 conn(0x7f63cc160a80 0x7f63cc0b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.924+0000 7f63ca000640 1 -- 172.21.15.26:0/2837282444 >> v1:172.21.15.86:6789/0 conn(0x7f63cc072490 legacy=0x7f63cc0b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:16.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.924+0000 7f63ca000640 1 -- 172.21.15.26:0/2837282444 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63cc073350 con 0x7f63cc160e60 2024-04-26T21:09:16.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.924+0000 7f63caa00640 1 --2- 172.21.15.26:0/2837282444 >> v2:172.21.15.86:3300/0 conn(0x7f63cc160a80 0x7f63cc0b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:16.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.925+0000 7f63ca000640 1 --2- 172.21.15.26:0/2837282444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63cc160e60 0x7f63cc06e210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f63cc051090 tx=0x7f63b402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354e92cf1dd716d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:16.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.925+0000 7f63c9600640 1 -- 172.21.15.26:0/2837282444 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b4037070 con 0x7f63cc160e60 2024-04-26T21:09:16.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.925+0000 7f63c9600640 1 -- 172.21.15.26:0/2837282444 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63b4032440 con 0x7f63cc160e60 2024-04-26T21:09:16.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.925+0000 7f63c9600640 1 -- 172.21.15.26:0/2837282444 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b4032810 con 0x7f63cc160e60 2024-04-26T21:09:16.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.926+0000 7f63cbe00640 1 -- 172.21.15.26:0/2837282444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63cc160e60 msgr2=0x7f63cc06e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:16.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.926+0000 7f63cbe00640 1 --2- 172.21.15.26:0/2837282444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63cc160e60 0x7f63cc06e210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f63cc051090 tx=0x7f63b402a230 comp rx=0 tx=0).stop 2024-04-26T21:09:16.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.926+0000 7f63cbe00640 1 -- 172.21.15.26:0/2837282444 shutdown_connections 2024-04-26T21:09:16.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.926+0000 7f63cbe00640 1 --2- 172.21.15.26:0/2837282444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63cc160e60 0x7f63cc06e210 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.926+0000 7f63cbe00640 1 --2- 172.21.15.26:0/2837282444 >> v2:172.21.15.86:3300/0 conn(0x7f63cc160a80 0x7f63cc0b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.926+0000 7f63cbe00640 1 -- 172.21.15.26:0/2837282444 >> 172.21.15.26:0/2837282444 conn(0x7f63cc0b2a10 msgr2=0x7f63cc0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:16.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.926+0000 7f63cbe00640 1 -- 172.21.15.26:0/2837282444 shutdown_connections 2024-04-26T21:09:16.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.926+0000 7f63cbe00640 1 -- 172.21.15.26:0/2837282444 wait complete. 2024-04-26T21:09:16.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.928+0000 7f63cbe00640 1 Processor -- start 2024-04-26T21:09:16.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.928+0000 7f63cbe00640 1 -- start start 2024-04-26T21:09:16.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.929+0000 7f63cbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63cc072490 0x7f63cc0d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:16.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.929+0000 7f63cbe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63cc160a80 0x7f63cc0aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:16.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.929+0000 7f63cbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63cc160e60 0x7f63cc147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:16.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.929+0000 7f63cbe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc1612d0 con 0x7f63cc160e60 2024-04-26T21:09:16.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.929+0000 7f63cbe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc0b5820 con 0x7f63cc072490 2024-04-26T21:09:16.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.929+0000 7f63cbe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f63cc072900 con 0x7f63cc160a80 2024-04-26T21:09:16.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.929+0000 7f63cb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63cc160e60 0x7f63cc147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:16.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.929+0000 7f63cb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63cc160e60 0x7f63cc147430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46824/0 (socket says 172.21.15.26:46824) 2024-04-26T21:09:16.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.929+0000 7f63cb400640 1 -- 172.21.15.26:0/2029957841 learned_addr learned my addr 172.21.15.26:0/2029957841 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:16.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.929+0000 7f63caa00640 1 --2- 172.21.15.26:0/2029957841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63cc072490 0x7f63cc0d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:16.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.930+0000 7f63ca000640 1 --2- 172.21.15.26:0/2029957841 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63cc160a80 0x7f63cc0aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:16.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.930+0000 7f63cb400640 1 -- 172.21.15.26:0/2029957841 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63cc160a80 msgr2=0x7f63cc0aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:16.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.930+0000 7f63cb400640 1 --2- 172.21.15.26:0/2029957841 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63cc160a80 0x7f63cc0aecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.930+0000 7f63cb400640 1 -- 172.21.15.26:0/2029957841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63cc072490 msgr2=0x7f63cc0d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:16.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.930+0000 7f63cb400640 1 --2- 172.21.15.26:0/2029957841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63cc072490 0x7f63cc0d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:16.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.930+0000 7f63cb400640 1 -- 172.21.15.26:0/2029957841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63cc1f8b40 con 0x7f63cc160e60 2024-04-26T21:09:16.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.930+0000 7f63cb400640 1 --2- 172.21.15.26:0/2029957841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63cc160e60 0x7f63cc147430 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f63c400c410 tx=0x7f63c400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:16.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.930+0000 7f63c2a00640 1 -- 172.21.15.26:0/2029957841 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c4011790 con 0x7f63cc160e60 2024-04-26T21:09:16.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.930+0000 7f63c2a00640 1 -- 172.21.15.26:0/2029957841 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63c4017070 con 0x7f63cc160e60 2024-04-26T21:09:16.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.930+0000 7f63c2a00640 1 -- 172.21.15.26:0/2029957841 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c4011bb0 con 0x7f63cc160e60 2024-04-26T21:09:16.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.930+0000 7f63cbe00640 1 -- 172.21.15.26:0/2029957841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63cc1f8e20 con 0x7f63cc160e60 2024-04-26T21:09:16.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.931+0000 7f63cbe00640 1 -- 172.21.15.26:0/2029957841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63cc1f93e0 con 0x7f63cc160e60 2024-04-26T21:09:16.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.932+0000 7f63cbe00640 1 -- 172.21.15.26:0/2029957841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63cc06a700 con 0x7f63cc160e60 2024-04-26T21:09:16.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.933+0000 7f63c2a00640 1 -- 172.21.15.26:0/2029957841 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f63c401c460 con 0x7f63cc160e60 2024-04-26T21:09:16.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.934+0000 7f63c2a00640 1 --2- 172.21.15.26:0/2029957841 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f63a806c8f0 0x7f63a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:16.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.935+0000 7f63c2a00640 1 -- 172.21.15.26:0/2029957841 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f63c408b730 con 0x7f63cc160e60 2024-04-26T21:09:16.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.935+0000 7f63caa00640 1 --2- 172.21.15.26:0/2029957841 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f63a806c8f0 0x7f63a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:16.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.936+0000 7f63caa00640 1 --2- 172.21.15.26:0/2029957841 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f63a806c8f0 0x7f63a806edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0075b0 tx=0x7f63bc007540 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:16.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:16.939+0000 7f63c2a00640 1 -- 172.21.15.26:0/2029957841 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f63c4021020 con 0x7f63cc160e60 2024-04-26T21:09:17.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.143+0000 7f63cbe00640 1 -- 172.21.15.26:0/2029957841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f63cc072270 con 0x7f63cc160e60 2024-04-26T21:09:17.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.144+0000 7f63c2a00640 1 -- 172.21.15.26:0/2029957841 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f63c4059b60 con 0x7f63cc160e60 2024-04-26T21:09:17.145 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-26T21:09:17.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.146+0000 7f63cbe00640 1 -- 172.21.15.26:0/2029957841 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f63a806c8f0 msgr2=0x7f63a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:17.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.146+0000 7f63cbe00640 1 --2- 172.21.15.26:0/2029957841 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f63a806c8f0 0x7f63a806edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0075b0 tx=0x7f63bc007540 comp rx=0 tx=0).stop 2024-04-26T21:09:17.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.146+0000 7f63cbe00640 1 -- 172.21.15.26:0/2029957841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63cc160e60 msgr2=0x7f63cc147430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:17.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.146+0000 7f63cbe00640 1 --2- 172.21.15.26:0/2029957841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63cc160e60 0x7f63cc147430 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f63c400c410 tx=0x7f63c400c8d0 comp rx=0 tx=0).stop 2024-04-26T21:09:17.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.147+0000 7f63cbe00640 1 -- 172.21.15.26:0/2029957841 shutdown_connections 2024-04-26T21:09:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.147+0000 7f63cbe00640 1 --2- 172.21.15.26:0/2029957841 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f63a806c8f0 0x7f63a806edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.147+0000 7f63cbe00640 1 --2- 172.21.15.26:0/2029957841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63cc160e60 0x7f63cc147430 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.147+0000 7f63cbe00640 1 --2- 172.21.15.26:0/2029957841 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63cc160a80 0x7f63cc0aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.147+0000 7f63cbe00640 1 --2- 172.21.15.26:0/2029957841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63cc072490 0x7f63cc0d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.147+0000 7f63cbe00640 1 -- 172.21.15.26:0/2029957841 >> 172.21.15.26:0/2029957841 conn(0x7f63cc0b2a10 msgr2=0x7f63cc152e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.147+0000 7f63cbe00640 1 -- 172.21.15.26:0/2029957841 shutdown_connections 2024-04-26T21:09:17.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.147+0000 7f63cbe00640 1 -- 172.21.15.26:0/2029957841 wait complete. 2024-04-26T21:09:17.169 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:09:17.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.380+0000 7fead1600640 1 Processor -- start 2024-04-26T21:09:17.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.380+0000 7fead1600640 1 -- start start 2024-04-26T21:09:17.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.382+0000 7fead1600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feacc146b90 0x7feacc06c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:17.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.382+0000 7fead1600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feacc0b6910 0x7feacc06c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:17.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.382+0000 7fead1600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feacc0b6ec0 con 0x7feacc161900 2024-04-26T21:09:17.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.382+0000 7fead1600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7feacc0b71a0 con 0x7feacc146b90 2024-04-26T21:09:17.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.382+0000 7fead1600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7feacc160e20 con 0x7feacc0b6910 2024-04-26T21:09:17.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.383+0000 7feacaa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feacc146b90 0x7feacc06c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:17.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.383+0000 7feacb400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feacc0b6910 0x7feacc06c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:17.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.383+0000 7feacaa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feacc146b90 0x7feacc06c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.26:55136/0 (socket says 172.21.15.26:55136) 2024-04-26T21:09:17.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.383+0000 7feacaa00640 1 -- 172.21.15.26:0/817209399 learned_addr learned my addr 172.21.15.26:0/817209399 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:17.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.383+0000 7feaca000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7feacc161900 0x7feacc0b6200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.26:42838/0 (socket says 172.21.15.26:42838) 2024-04-26T21:09:17.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.384+0000 7feacaa00640 1 -- 172.21.15.26:0/817209399 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feacc0b6910 msgr2=0x7feacc06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:17.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.384+0000 7feacaa00640 1 --2- 172.21.15.26:0/817209399 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feacc0b6910 0x7feacc06c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:17.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.384+0000 7feacaa00640 1 -- 172.21.15.26:0/817209399 >> v1:172.21.15.86:6789/0 conn(0x7feacc161900 legacy=0x7feacc0b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:17.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.384+0000 7feacaa00640 1 -- 172.21.15.26:0/817209399 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feacc161100 con 0x7feacc146b90 2024-04-26T21:09:17.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.384+0000 7feacb400640 1 --2- 172.21.15.26:0/817209399 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feacc0b6910 0x7feacc06c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:09:17.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.385+0000 7feacaa00640 1 --2- 172.21.15.26:0/817209399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feacc146b90 0x7feacc06c240 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feabc009870 tx=0x7feabc02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18b4ab7697f78ebf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:17.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.386+0000 7feac9600640 1 -- 172.21.15.26:0/817209399 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc037070 con 0x7feacc146b90 2024-04-26T21:09:17.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.386+0000 7feac9600640 1 -- 172.21.15.26:0/817209399 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feabc032440 con 0x7feacc146b90 2024-04-26T21:09:17.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.386+0000 7feac9600640 1 -- 172.21.15.26:0/817209399 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc032810 con 0x7feacc146b90 2024-04-26T21:09:17.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.387+0000 7fead1600640 1 -- 172.21.15.26:0/817209399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feacc146b90 msgr2=0x7feacc06c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:17.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.387+0000 7fead1600640 1 --2- 172.21.15.26:0/817209399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feacc146b90 0x7feacc06c240 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feabc009870 tx=0x7feabc02a250 comp rx=0 tx=0).stop 2024-04-26T21:09:17.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.387+0000 7fead1600640 1 -- 172.21.15.26:0/817209399 shutdown_connections 2024-04-26T21:09:17.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.387+0000 7fead1600640 1 --2- 172.21.15.26:0/817209399 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feacc0b6910 0x7feacc06c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:17.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.387+0000 7fead1600640 1 --2- 172.21.15.26:0/817209399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feacc146b90 0x7feacc06c240 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:17.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.387+0000 7fead1600640 1 -- 172.21.15.26:0/817209399 >> 172.21.15.26:0/817209399 conn(0x7feacc0b3be0 msgr2=0x7feacc0b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:17.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.387+0000 7fead1600640 1 -- 172.21.15.26:0/817209399 shutdown_connections 2024-04-26T21:09:17.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.387+0000 7fead1600640 1 -- 172.21.15.26:0/817209399 wait complete. 2024-04-26T21:09:17.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.389+0000 7fead1600640 1 Processor -- start 2024-04-26T21:09:17.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.389+0000 7fead1600640 1 -- start start 2024-04-26T21:09:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.390+0000 7fead1600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feacc0b6910 0x7feacc145900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.390+0000 7fead1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feacc146b90 0x7feacc15c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.390+0000 7fead1600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feacc161900 0x7feacc15e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.390+0000 7fead1600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7feacc070800 con 0x7feacc161900 2024-04-26T21:09:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.390+0000 7fead1600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feacc070680 con 0x7feacc146b90 2024-04-26T21:09:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.390+0000 7fead1600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7feacc070980 con 0x7feacc0b6910 2024-04-26T21:09:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.390+0000 7feacb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feacc161900 0x7feacc15e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.390+0000 7feacb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feacc161900 0x7feacc15e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46836/0 (socket says 172.21.15.26:46836) 2024-04-26T21:09:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.390+0000 7feacb400640 1 -- 172.21.15.26:0/3599507394 learned_addr learned my addr 172.21.15.26:0/3599507394 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.391+0000 7feacaa00640 1 --2- 172.21.15.26:0/3599507394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feacc0b6910 0x7feacc145900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.391+0000 7feacb400640 1 -- 172.21.15.26:0/3599507394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feacc0b6910 msgr2=0x7feacc145900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.391+0000 7feacb400640 1 --2- 172.21.15.26:0/3599507394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feacc0b6910 0x7feacc145900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.391+0000 7feacb400640 1 -- 172.21.15.26:0/3599507394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feacc146b90 msgr2=0x7feacc15c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.391+0000 7feacb400640 1 --2- 172.21.15.26:0/3599507394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feacc146b90 0x7feacc15c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.391+0000 7feacb400640 1 -- 172.21.15.26:0/3599507394 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feacc144860 con 0x7feacc161900 2024-04-26T21:09:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.391+0000 7feacb400640 1 --2- 172.21.15.26:0/3599507394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feacc161900 0x7feacc15e3c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7feac4009f00 tx=0x7feac400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:17.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.391+0000 7feac2a00640 1 -- 172.21.15.26:0/3599507394 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4018030 con 0x7feacc161900 2024-04-26T21:09:17.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.391+0000 7feac2a00640 1 -- 172.21.15.26:0/3599507394 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feac400ce50 con 0x7feacc161900 2024-04-26T21:09:17.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.391+0000 7feac2a00640 1 -- 172.21.15.26:0/3599507394 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4003bb0 con 0x7feacc161900 2024-04-26T21:09:17.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.391+0000 7fead1600640 1 -- 172.21.15.26:0/3599507394 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feacc144b40 con 0x7feacc161900 2024-04-26T21:09:17.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.392+0000 7fead1600640 1 -- 172.21.15.26:0/3599507394 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feacc1453e0 con 0x7feacc161900 2024-04-26T21:09:17.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.393+0000 7fead1600640 1 -- 172.21.15.26:0/3599507394 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feacc0693d0 con 0x7feacc161900 2024-04-26T21:09:17.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.394+0000 7feac2a00640 1 -- 172.21.15.26:0/3599507394 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feac401b070 con 0x7feacc161900 2024-04-26T21:09:17.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.395+0000 7feac2a00640 1 --2- 172.21.15.26:0/3599507394 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7feaac06c8f0 0x7feaac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:17.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.395+0000 7feac2a00640 1 -- 172.21.15.26:0/3599507394 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7feac408c530 con 0x7feacc161900 2024-04-26T21:09:17.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.395+0000 7feacaa00640 1 --2- 172.21.15.26:0/3599507394 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7feaac06c8f0 0x7feaac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:17.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.396+0000 7feacaa00640 1 --2- 172.21.15.26:0/3599507394 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7feaac06c8f0 0x7feaac06edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feabc034000 tx=0x7feabc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:17.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.400+0000 7feac2a00640 1 -- 172.21.15.26:0/3599507394 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feac4010040 con 0x7feacc161900 2024-04-26T21:09:17.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.606+0000 7fead1600640 1 -- 172.21.15.26:0/3599507394 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feacc144d10 con 0x7feacc161900 2024-04-26T21:09:17.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.607+0000 7feac2a00640 1 -- 172.21.15.26:0/3599507394 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7feac4012340 con 0x7feacc161900 2024-04-26T21:09:17.610 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-26T21:09:17.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.609+0000 7fead1600640 1 -- 172.21.15.26:0/3599507394 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7feaac06c8f0 msgr2=0x7feaac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:17.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.609+0000 7fead1600640 1 --2- 172.21.15.26:0/3599507394 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7feaac06c8f0 0x7feaac06edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feabc034000 tx=0x7feabc02e000 comp rx=0 tx=0).stop 2024-04-26T21:09:17.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.609+0000 7fead1600640 1 -- 172.21.15.26:0/3599507394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feacc161900 msgr2=0x7feacc15e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:17.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.609+0000 7fead1600640 1 --2- 172.21.15.26:0/3599507394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feacc161900 0x7feacc15e3c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7feac4009f00 tx=0x7feac400c490 comp rx=0 tx=0).stop 2024-04-26T21:09:17.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.609+0000 7fead1600640 1 -- 172.21.15.26:0/3599507394 shutdown_connections 2024-04-26T21:09:17.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.609+0000 7fead1600640 1 --2- 172.21.15.26:0/3599507394 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7feaac06c8f0 0x7feaac06edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:17.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.609+0000 7fead1600640 1 --2- 172.21.15.26:0/3599507394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feacc161900 0x7feacc15e3c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:17.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.609+0000 7fead1600640 1 --2- 172.21.15.26:0/3599507394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feacc146b90 0x7feacc15c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:17.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.609+0000 7fead1600640 1 --2- 172.21.15.26:0/3599507394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feacc0b6910 0x7feacc145900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:17.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.609+0000 7fead1600640 1 -- 172.21.15.26:0/3599507394 >> 172.21.15.26:0/3599507394 conn(0x7feacc0b3be0 msgr2=0x7feacc06c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:17.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.610+0000 7fead1600640 1 -- 172.21.15.26:0/3599507394 shutdown_connections 2024-04-26T21:09:17.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.610+0000 7fead1600640 1 -- 172.21.15.26:0/3599507394 wait complete. 2024-04-26T21:09:17.630 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:09:17.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.850+0000 7f85c0a00640 1 Processor -- start 2024-04-26T21:09:17.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.850+0000 7f85c0a00640 1 -- start start 2024-04-26T21:09:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.851+0000 7f85c0a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85bc0b74d0 0x7f85bc0d0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.851+0000 7f85c0a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f85bc0d0ab0 0x7f85bc0d3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.851+0000 7f85c0a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85bc0b7f90 con 0x7f85bc0d39c0 2024-04-26T21:09:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.851+0000 7f85c0a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f85bc0b8270 con 0x7f85bc0d0ab0 2024-04-26T21:09:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.851+0000 7f85c0a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f85bc0b8550 con 0x7f85bc0b74d0 2024-04-26T21:09:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.852+0000 7f85baa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f85bc0d0ab0 0x7f85bc0d3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.852+0000 7f85baa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f85bc0d0ab0 0x7f85bc0d3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46848/0 (socket says 172.21.15.26:46848) 2024-04-26T21:09:17.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.852+0000 7f85baa00640 1 -- 172.21.15.26:0/735415903 learned_addr learned my addr 172.21.15.26:0/735415903 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:17.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.852+0000 7f85bb400640 1 --2- 172.21.15.26:0/735415903 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85bc0b74d0 0x7f85bc0d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:17.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.852+0000 7f85baa00640 1 -- 172.21.15.26:0/735415903 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85bc0b74d0 msgr2=0x7f85bc0d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:17.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.852+0000 7f85baa00640 1 --2- 172.21.15.26:0/735415903 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85bc0b74d0 0x7f85bc0d0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:17.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.852+0000 7f85baa00640 1 -- 172.21.15.26:0/735415903 >> v1:172.21.15.86:6789/0 conn(0x7f85bc0d39c0 legacy=0x7f85bc15eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:17.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.852+0000 7f85baa00640 1 -- 172.21.15.26:0/735415903 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85bc0b8830 con 0x7f85bc0d0ab0 2024-04-26T21:09:17.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.853+0000 7f85baa00640 1 --2- 172.21.15.26:0/735415903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85bc0d0ab0 0x7f85bc0d3480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009e50 tx=0x7f85a402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5401408cec66afe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:17.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.854+0000 7f85ba000640 1 -- 172.21.15.26:0/735415903 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4036070 con 0x7f85bc0d0ab0 2024-04-26T21:09:17.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.854+0000 7f85ba000640 1 -- 172.21.15.26:0/735415903 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a4031600 con 0x7f85bc0d0ab0 2024-04-26T21:09:17.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.854+0000 7f85ba000640 1 -- 172.21.15.26:0/735415903 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a40319d0 con 0x7f85bc0d0ab0 2024-04-26T21:09:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.854+0000 7f85c0a00640 1 -- 172.21.15.26:0/735415903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85bc0d0ab0 msgr2=0x7f85bc0d3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.854+0000 7f85c0a00640 1 --2- 172.21.15.26:0/735415903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85bc0d0ab0 0x7f85bc0d3480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009e50 tx=0x7f85a402a710 comp rx=0 tx=0).stop 2024-04-26T21:09:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.855+0000 7f85c0a00640 1 -- 172.21.15.26:0/735415903 shutdown_connections 2024-04-26T21:09:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.855+0000 7f85c0a00640 1 --2- 172.21.15.26:0/735415903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85bc0d0ab0 0x7f85bc0d3480 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:17.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.855+0000 7f85c0a00640 1 --2- 172.21.15.26:0/735415903 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85bc0b74d0 0x7f85bc0d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:17.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.855+0000 7f85c0a00640 1 -- 172.21.15.26:0/735415903 >> 172.21.15.26:0/735415903 conn(0x7f85bc0d4f30 msgr2=0x7f85bc0d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:17.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.855+0000 7f85c0a00640 1 -- 172.21.15.26:0/735415903 shutdown_connections 2024-04-26T21:09:17.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.855+0000 7f85c0a00640 1 -- 172.21.15.26:0/735415903 wait complete. 2024-04-26T21:09:17.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.856+0000 7f85c0a00640 1 Processor -- start 2024-04-26T21:09:17.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.857+0000 7f85c0a00640 1 -- start start 2024-04-26T21:09:17.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.857+0000 7f85c0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85bc0b74d0 0x7f85bc09f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:17.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.857+0000 7f85c0a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85bc0d0ab0 0x7f85bc0a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:17.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.857+0000 7f85c0a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85bc0d39c0 0x7f85bc160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:17.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.857+0000 7f85c0a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc15f290 con 0x7f85bc0d39c0 2024-04-26T21:09:17.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.857+0000 7f85c0a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc0b78b0 con 0x7f85bc0b74d0 2024-04-26T21:09:17.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.857+0000 7f85c0a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f85bc15f410 con 0x7f85bc0d0ab0 2024-04-26T21:09:17.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.858+0000 7f85bbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85bc0d39c0 0x7f85bc160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:17.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.858+0000 7f85bbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85bc0d39c0 0x7f85bc160bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46858/0 (socket says 172.21.15.26:46858) 2024-04-26T21:09:17.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.858+0000 7f85bbe00640 1 -- 172.21.15.26:0/1241376504 learned_addr learned my addr 172.21.15.26:0/1241376504 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:17.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.858+0000 7f85bb400640 1 --2- 172.21.15.26:0/1241376504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85bc0b74d0 0x7f85bc09f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:17.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.858+0000 7f85baa00640 1 --2- 172.21.15.26:0/1241376504 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85bc0d0ab0 0x7f85bc0a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:17.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.858+0000 7f85bbe00640 1 -- 172.21.15.26:0/1241376504 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85bc0d0ab0 msgr2=0x7f85bc0a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:17.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.858+0000 7f85bbe00640 1 --2- 172.21.15.26:0/1241376504 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85bc0d0ab0 0x7f85bc0a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:17.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.858+0000 7f85bbe00640 1 -- 172.21.15.26:0/1241376504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85bc0b74d0 msgr2=0x7f85bc09f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:17.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.858+0000 7f85bbe00640 1 --2- 172.21.15.26:0/1241376504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85bc0b74d0 0x7f85bc09f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:17.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.859+0000 7f85bbe00640 1 -- 172.21.15.26:0/1241376504 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85bc0a13f0 con 0x7f85bc0d39c0 2024-04-26T21:09:17.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.859+0000 7f85bbe00640 1 --2- 172.21.15.26:0/1241376504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85bc0d39c0 0x7f85bc160bf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f85b400c3c0 tx=0x7f85b400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:17.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.859+0000 7f85b3400640 1 -- 172.21.15.26:0/1241376504 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b40052d0 con 0x7f85bc0d39c0 2024-04-26T21:09:17.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.859+0000 7f85b3400640 1 -- 172.21.15.26:0/1241376504 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85b4005470 con 0x7f85bc0d39c0 2024-04-26T21:09:17.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.859+0000 7f85b3400640 1 -- 172.21.15.26:0/1241376504 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b4007240 con 0x7f85bc0d39c0 2024-04-26T21:09:17.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.859+0000 7f85c0a00640 1 -- 172.21.15.26:0/1241376504 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85bc0a16d0 con 0x7f85bc0d39c0 2024-04-26T21:09:17.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.860+0000 7f85c0a00640 1 -- 172.21.15.26:0/1241376504 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85bc0a1c90 con 0x7f85bc0d39c0 2024-04-26T21:09:17.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.861+0000 7f85c0a00640 1 -- 172.21.15.26:0/1241376504 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85bc0506c0 con 0x7f85bc0d39c0 2024-04-26T21:09:17.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.861+0000 7f85b3400640 1 -- 172.21.15.26:0/1241376504 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f85b40073e0 con 0x7f85bc0d39c0 2024-04-26T21:09:17.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.863+0000 7f85b3400640 1 --2- 172.21.15.26:0/1241376504 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f859806c8f0 0x7f859806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:17.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.863+0000 7f85b3400640 1 -- 172.21.15.26:0/1241376504 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f85b408afb0 con 0x7f85bc0d39c0 2024-04-26T21:09:17.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.863+0000 7f85bb400640 1 --2- 172.21.15.26:0/1241376504 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f859806c8f0 0x7f859806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:17.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.864+0000 7f85bb400640 1 --2- 172.21.15.26:0/1241376504 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f859806c8f0 0x7f859806edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f85ac006fd0 tx=0x7f85ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:17.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:17.867+0000 7f85b3400640 1 -- 172.21.15.26:0/1241376504 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f85b4059340 con 0x7f85bc0d39c0 2024-04-26T21:09:18.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.070+0000 7f85c0a00640 1 -- 172.21.15.26:0/1241376504 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f85bc074690 con 0x7f85bc0d39c0 2024-04-26T21:09:18.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.071+0000 7f85b3400640 1 -- 172.21.15.26:0/1241376504 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f85b4058c40 con 0x7f85bc0d39c0 2024-04-26T21:09:18.073 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-26T21:09:18.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.074+0000 7f85c0a00640 1 -- 172.21.15.26:0/1241376504 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f859806c8f0 msgr2=0x7f859806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:18.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.074+0000 7f85c0a00640 1 --2- 172.21.15.26:0/1241376504 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f859806c8f0 0x7f859806edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f85ac006fd0 tx=0x7f85ac008040 comp rx=0 tx=0).stop 2024-04-26T21:09:18.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.074+0000 7f85c0a00640 1 -- 172.21.15.26:0/1241376504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85bc0d39c0 msgr2=0x7f85bc160bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:18.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.074+0000 7f85c0a00640 1 --2- 172.21.15.26:0/1241376504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85bc0d39c0 0x7f85bc160bf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f85b400c3c0 tx=0x7f85b400c880 comp rx=0 tx=0).stop 2024-04-26T21:09:18.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.074+0000 7f85c0a00640 1 -- 172.21.15.26:0/1241376504 shutdown_connections 2024-04-26T21:09:18.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.074+0000 7f85c0a00640 1 --2- 172.21.15.26:0/1241376504 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f859806c8f0 0x7f859806edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.074+0000 7f85c0a00640 1 --2- 172.21.15.26:0/1241376504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85bc0d39c0 0x7f85bc160bf0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.074+0000 7f85c0a00640 1 --2- 172.21.15.26:0/1241376504 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85bc0d0ab0 0x7f85bc0a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.074+0000 7f85c0a00640 1 --2- 172.21.15.26:0/1241376504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85bc0b74d0 0x7f85bc09f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.074+0000 7f85c0a00640 1 -- 172.21.15.26:0/1241376504 >> 172.21.15.26:0/1241376504 conn(0x7f85bc0d4f30 msgr2=0x7f85bc0d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:18.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.075+0000 7f85c0a00640 1 -- 172.21.15.26:0/1241376504 shutdown_connections 2024-04-26T21:09:18.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.075+0000 7f85c0a00640 1 -- 172.21.15.26:0/1241376504 wait complete. 2024-04-26T21:09:18.119 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace (tasks.cephfs.test_backtrace.TestBacktrace) That the 'parent' 'layout' and 'symlink' xattrs on the head objects of files 2024-04-26T21:09:18.119 INFO:tasks.cephfs_test_runner:test_backtrace (tasks.cephfs.test_backtrace.TestBacktrace) 2024-04-26T21:09:18.119 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:09:18.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.346+0000 7f90dd400640 1 Processor -- start 2024-04-26T21:09:18.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.346+0000 7f90dd400640 1 -- start start 2024-04-26T21:09:18.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.347+0000 7f90dd400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f90d80d1d60 0x7f90d80b7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:18.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.347+0000 7f90dd400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f90d80b8470 0x7f90d8162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:18.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.347+0000 7f90dd400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90d80d3d80 con 0x7f90d80d1980 2024-04-26T21:09:18.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.348+0000 7f90dd400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f90d80d04e0 con 0x7f90d80d1d60 2024-04-26T21:09:18.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.348+0000 7f90dd400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f90d80cfe90 con 0x7f90d80b8470 2024-04-26T21:09:18.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.348+0000 7f90d7400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f90d80d1d60 0x7f90d80b7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:18.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.348+0000 7f90d7e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f90d80d1980 0x7f90d80d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45404/0 (socket says 172.21.15.26:45404) 2024-04-26T21:09:18.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.348+0000 7f90d7400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f90d80d1d60 0x7f90d80b7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46860/0 (socket says 172.21.15.26:46860) 2024-04-26T21:09:18.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.348+0000 7f90d7e00640 1 -- 172.21.15.26:0/1542179879 learned_addr learned my addr 172.21.15.26:0/1542179879 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:18.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.348+0000 7f90dca00640 1 --2- 172.21.15.26:0/1542179879 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f90d80b8470 0x7f90d8162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:18.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.348+0000 7f90d7400640 1 -- 172.21.15.26:0/1542179879 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f90d80b8470 msgr2=0x7f90d8162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:18.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.348+0000 7f90d7400640 1 --2- 172.21.15.26:0/1542179879 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f90d80b8470 0x7f90d8162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.348+0000 7f90d7400640 1 -- 172.21.15.26:0/1542179879 >> v1:172.21.15.26:6789/0 conn(0x7f90d80d1980 legacy=0x7f90d80d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:18.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.348+0000 7f90d7400640 1 -- 172.21.15.26:0/1542179879 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d80d0170 con 0x7f90d80d1d60 2024-04-26T21:09:18.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.349+0000 7f90dca00640 1 --2- 172.21.15.26:0/1542179879 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f90d80b8470 0x7f90d8162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:18.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.350+0000 7f90d7400640 1 --2- 172.21.15.26:0/1542179879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d80d1d60 0x7f90d80b7f30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f90c00097a0 tx=0x7f90c002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c49f16c83ebb420 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:18.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.350+0000 7f90d6a00640 1 -- 172.21.15.26:0/1542179879 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0037070 con 0x7f90d80d1d60 2024-04-26T21:09:18.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.350+0000 7f90d6a00640 1 -- 172.21.15.26:0/1542179879 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c0032440 con 0x7f90d80d1d60 2024-04-26T21:09:18.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.350+0000 7f90d6a00640 1 -- 172.21.15.26:0/1542179879 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0032810 con 0x7f90d80d1d60 2024-04-26T21:09:18.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.350+0000 7f90dd400640 1 -- 172.21.15.26:0/1542179879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d80d1d60 msgr2=0x7f90d80b7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:18.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.350+0000 7f90dd400640 1 --2- 172.21.15.26:0/1542179879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d80d1d60 0x7f90d80b7f30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f90c00097a0 tx=0x7f90c002a180 comp rx=0 tx=0).stop 2024-04-26T21:09:18.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.351+0000 7f90dd400640 1 -- 172.21.15.26:0/1542179879 shutdown_connections 2024-04-26T21:09:18.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.351+0000 7f90dd400640 1 --2- 172.21.15.26:0/1542179879 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f90d80b8470 0x7f90d8162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.351+0000 7f90dd400640 1 --2- 172.21.15.26:0/1542179879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d80d1d60 0x7f90d80b7f30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.351+0000 7f90dd400640 1 -- 172.21.15.26:0/1542179879 >> 172.21.15.26:0/1542179879 conn(0x7f90d809e410 msgr2=0x7f90d80d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:18.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.351+0000 7f90dd400640 1 -- 172.21.15.26:0/1542179879 shutdown_connections 2024-04-26T21:09:18.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.351+0000 7f90dd400640 1 -- 172.21.15.26:0/1542179879 wait complete. 2024-04-26T21:09:18.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.352+0000 7f90dd400640 1 Processor -- start 2024-04-26T21:09:18.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.353+0000 7f90dd400640 1 -- start start 2024-04-26T21:09:18.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.353+0000 7f90dd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90d80b8470 0x7f90d80cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:18.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.353+0000 7f90dd400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d80d1980 0x7f90d81572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:18.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.353+0000 7f90dd400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f90d80d1d60 0x7f90d80a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:18.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.353+0000 7f90dd400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f90d80d41e0 con 0x7f90d80d1980 2024-04-26T21:09:18.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.353+0000 7f90dd400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90d80d4060 con 0x7f90d80b8470 2024-04-26T21:09:18.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.353+0000 7f90dd400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f90d8162ab0 con 0x7f90d80d1d60 2024-04-26T21:09:18.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.354+0000 7f90d7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d80d1980 0x7f90d81572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:18.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.354+0000 7f90d7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d80d1980 0x7f90d81572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46868/0 (socket says 172.21.15.26:46868) 2024-04-26T21:09:18.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.354+0000 7f90d7400640 1 -- 172.21.15.26:0/858544696 learned_addr learned my addr 172.21.15.26:0/858544696 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:18.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.354+0000 7f90d7e00640 1 --2- 172.21.15.26:0/858544696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90d80b8470 0x7f90d80cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:18.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.354+0000 7f90dca00640 1 --2- 172.21.15.26:0/858544696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f90d80d1d60 0x7f90d80a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:18.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.354+0000 7f90d7400640 1 -- 172.21.15.26:0/858544696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f90d80d1d60 msgr2=0x7f90d80a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:18.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.354+0000 7f90d7400640 1 --2- 172.21.15.26:0/858544696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f90d80d1d60 0x7f90d80a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.354+0000 7f90d7400640 1 -- 172.21.15.26:0/858544696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90d80b8470 msgr2=0x7f90d80cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:18.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.354+0000 7f90d7400640 1 --2- 172.21.15.26:0/858544696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90d80b8470 0x7f90d80cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.354+0000 7f90d7400640 1 -- 172.21.15.26:0/858544696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d80ccab0 con 0x7f90d80d1980 2024-04-26T21:09:18.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.354+0000 7f90dca00640 1 --2- 172.21.15.26:0/858544696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f90d80d1d60 0x7f90d80a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:18.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.354+0000 7f90d7e00640 1 --2- 172.21.15.26:0/858544696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90d80b8470 0x7f90d80cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:18.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.355+0000 7f90d7400640 1 --2- 172.21.15.26:0/858544696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d80d1980 0x7f90d81572e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f90c0035040 tx=0x7f90c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:18.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.355+0000 7f90cfe00640 1 -- 172.21.15.26:0/858544696 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0037040 con 0x7f90d80d1980 2024-04-26T21:09:18.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.355+0000 7f90cfe00640 1 -- 172.21.15.26:0/858544696 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c0030770 con 0x7f90d80d1980 2024-04-26T21:09:18.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.355+0000 7f90cfe00640 1 -- 172.21.15.26:0/858544696 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0030b80 con 0x7f90d80d1980 2024-04-26T21:09:18.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.355+0000 7f90dd400640 1 -- 172.21.15.26:0/858544696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d80ccd90 con 0x7f90d80d1980 2024-04-26T21:09:18.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.355+0000 7f90dd400640 1 -- 172.21.15.26:0/858544696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d80cd630 con 0x7f90d80d1980 2024-04-26T21:09:18.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.356+0000 7f90dd400640 1 -- 172.21.15.26:0/858544696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90d80cd180 con 0x7f90d80d1980 2024-04-26T21:09:18.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.357+0000 7f90cfe00640 1 -- 172.21.15.26:0/858544696 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90c0044020 con 0x7f90d80d1980 2024-04-26T21:09:18.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.359+0000 7f90cfe00640 1 --2- 172.21.15.26:0/858544696 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f90b406c8f0 0x7f90b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:18.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.359+0000 7f90cfe00640 1 -- 172.21.15.26:0/858544696 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f90c00ac150 con 0x7f90d80d1980 2024-04-26T21:09:18.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.359+0000 7f90d7e00640 1 --2- 172.21.15.26:0/858544696 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f90b406c8f0 0x7f90b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:18.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.360+0000 7f90d7e00640 1 --2- 172.21.15.26:0/858544696 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f90b406c8f0 0x7f90b406edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f90c8000f60 tx=0x7f90c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:18.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.364+0000 7f90cfe00640 1 -- 172.21.15.26:0/858544696 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f90c0032e20 con 0x7f90d80d1980 2024-04-26T21:09:18.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.569+0000 7f90dd400640 1 -- 172.21.15.26:0/858544696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f90d80693d0 con 0x7f90d80d1980 2024-04-26T21:09:18.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.571+0000 7f90cfe00640 1 -- 172.21.15.26:0/858544696 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16557 (secure 0 0 0) 0x7f90c007a570 con 0x7f90d80d1980 2024-04-26T21:09:18.573 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:18.573 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:08:39.177854+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:08:35.481169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:08:36.609498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:09:18.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.574+0000 7f90dd400640 1 -- 172.21.15.26:0/858544696 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f90b406c8f0 msgr2=0x7f90b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:18.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.574+0000 7f90dd400640 1 --2- 172.21.15.26:0/858544696 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f90b406c8f0 0x7f90b406edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f90c8000f60 tx=0x7f90c8009040 comp rx=0 tx=0).stop 2024-04-26T21:09:18.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.574+0000 7f90dd400640 1 -- 172.21.15.26:0/858544696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d80d1980 msgr2=0x7f90d81572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:18.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.574+0000 7f90dd400640 1 --2- 172.21.15.26:0/858544696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d80d1980 0x7f90d81572e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f90c0035040 tx=0x7f90c00046c0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.574+0000 7f90dd400640 1 -- 172.21.15.26:0/858544696 shutdown_connections 2024-04-26T21:09:18.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.574+0000 7f90dd400640 1 --2- 172.21.15.26:0/858544696 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f90b406c8f0 0x7f90b406edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.574+0000 7f90dd400640 1 --2- 172.21.15.26:0/858544696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f90d80d1d60 0x7f90d80a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.574+0000 7f90dd400640 1 --2- 172.21.15.26:0/858544696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90d80d1980 0x7f90d81572e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.574+0000 7f90dd400640 1 --2- 172.21.15.26:0/858544696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90d80b8470 0x7f90d80cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.574+0000 7f90dd400640 1 -- 172.21.15.26:0/858544696 >> 172.21.15.26:0/858544696 conn(0x7f90d809e410 msgr2=0x7f90d806cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:18.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.574+0000 7f90dd400640 1 -- 172.21.15.26:0/858544696 shutdown_connections 2024-04-26T21:09:18.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.574+0000 7f90dd400640 1 -- 172.21.15.26:0/858544696 wait complete. 2024-04-26T21:09:18.595 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:08:31.322511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:08:35.481169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:08:36.609498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:09:18.596 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:09:18.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.800+0000 7f08e7200640 1 Processor -- start 2024-04-26T21:09:18.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.801+0000 7f08e7200640 1 -- start start 2024-04-26T21:09:18.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.802+0000 7f08e7200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f08e8162e60 0x7f08e81632e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:18.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.802+0000 7f08e7200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08e80d3870 con 0x7f08e815f290 2024-04-26T21:09:18.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.802+0000 7f08e7200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08e80d3b50 con 0x7f08e80b8640 2024-04-26T21:09:18.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.802+0000 7f08e7200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f08e80d3e30 con 0x7f08e8162e60 2024-04-26T21:09:18.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.803+0000 7f08e5e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f08e815f290 0x7f08e80d5700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45418/0 (socket says 172.21.15.26:45418) 2024-04-26T21:09:18.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.803+0000 7f08e5e00640 1 -- 172.21.15.26:0/167804909 learned_addr learned my addr 172.21.15.26:0/167804909 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.803+0000 7f08e5400640 1 --2- 172.21.15.26:0/167804909 >> v2:172.21.15.86:3300/0 conn(0x7f08e8162e60 0x7f08e81632e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.803+0000 7f08e4a00640 1 -- 172.21.15.26:0/167804909 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1415841899 0 0) 0x7f08e80d3870 con 0x7f08e815f290 2024-04-26T21:09:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.803+0000 7f08e4a00640 1 -- 172.21.15.26:0/167804909 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08cc003520 con 0x7f08e815f290 2024-04-26T21:09:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.803+0000 7f08e4a00640 1 -- 172.21.15.26:0/167804909 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3372971224 0 0) 0x7f08cc003520 con 0x7f08e815f290 2024-04-26T21:09:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.804+0000 7f08e4a00640 1 -- 172.21.15.26:0/167804909 >> v2:172.21.15.86:3300/0 conn(0x7f08e8162e60 msgr2=0x7f08e81632e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.804+0000 7f08e4a00640 1 --2- 172.21.15.26:0/167804909 >> v2:172.21.15.86:3300/0 conn(0x7f08e8162e60 0x7f08e81632e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.804+0000 7f08e4a00640 1 -- 172.21.15.26:0/167804909 >> v1:172.21.15.86:6789/0 conn(0x7f08e80b8640 legacy=0x7f08e80b8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.804+0000 7f08e4a00640 1 -- 172.21.15.26:0/167804909 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e80b7e40 con 0x7f08e815f290 2024-04-26T21:09:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.804+0000 7f08e4a00640 1 -- 172.21.15.26:0/167804909 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f08d80019f0 con 0x7f08e815f290 2024-04-26T21:09:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.804+0000 7f08e4a00640 1 -- 172.21.15.26:0/167804909 >> v1:172.21.15.26:6789/0 conn(0x7f08e815f290 legacy=0x7f08e80d5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.804+0000 7f08e4a00640 1 --2- 172.21.15.26:0/167804909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08cc0039b0 0x7f08cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.804+0000 7f08e4a00640 1 --2- 172.21.15.26:0/167804909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08cc004800 0x7f08cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.804+0000 7f08e4a00640 1 --2- 172.21.15.26:0/167804909 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08cc0051c0 0x7f08cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.804+0000 7f08e4a00640 1 -- 172.21.15.26:0/167804909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f08e80d3e30 con 0x7f08cc0039b0 2024-04-26T21:09:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.804+0000 7f08e4a00640 1 -- 172.21.15.26:0/167804909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc00c060 con 0x7f08cc004800 2024-04-26T21:09:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.804+0000 7f08e4a00640 1 -- 172.21.15.26:0/167804909 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f08cc00c260 con 0x7f08cc0051c0 2024-04-26T21:09:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.804+0000 7f08e5e00640 1 --2- 172.21.15.26:0/167804909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08cc0039b0 0x7f08cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.804+0000 7f08e5400640 1 --2- 172.21.15.26:0/167804909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08cc004800 0x7f08cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.805+0000 7f08e6800640 1 --2- 172.21.15.26:0/167804909 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08cc0051c0 0x7f08cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.805+0000 7f08e5e00640 1 -- 172.21.15.26:0/167804909 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08cc0051c0 msgr2=0x7f08cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.805+0000 7f08e5e00640 1 --2- 172.21.15.26:0/167804909 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08cc0051c0 0x7f08cc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.805+0000 7f08e5e00640 1 -- 172.21.15.26:0/167804909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08cc004800 msgr2=0x7f08cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.805+0000 7f08e5e00640 1 --2- 172.21.15.26:0/167804909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08cc004800 0x7f08cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.805+0000 7f08e6800640 1 -- 172.21.15.26:0/167804909 reap_dead start 2024-04-26T21:09:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.805+0000 7f08e5e00640 1 -- 172.21.15.26:0/167804909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08cc003520 con 0x7f08cc0039b0 2024-04-26T21:09:18.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.806+0000 7f08e5e00640 1 --2- 172.21.15.26:0/167804909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08cc0039b0 0x7f08cc003dd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f08d800a850 tx=0x7f08cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c7893026e1f256 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.806+0000 7f08e4a00640 1 -- 172.21.15.26:0/167804909 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d800a4a0 con 0x7f08cc0039b0 2024-04-26T21:09:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.806+0000 7f08e4a00640 1 -- 172.21.15.26:0/167804909 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d802f070 con 0x7f08cc0039b0 2024-04-26T21:09:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.806+0000 7f08e4a00640 1 -- 172.21.15.26:0/167804909 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d802bd20 con 0x7f08cc0039b0 2024-04-26T21:09:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.807+0000 7f08e7200640 1 -- 172.21.15.26:0/167804909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08cc0039b0 msgr2=0x7f08cc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.807+0000 7f08e7200640 1 --2- 172.21.15.26:0/167804909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08cc0039b0 0x7f08cc003dd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f08d800a850 tx=0x7f08cc005830 comp rx=0 tx=0).stop 2024-04-26T21:09:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.807+0000 7f08e7200640 1 -- 172.21.15.26:0/167804909 shutdown_connections 2024-04-26T21:09:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.807+0000 7f08e7200640 1 --2- 172.21.15.26:0/167804909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08cc0039b0 0x7f08cc003dd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.807+0000 7f08e7200640 1 -- 172.21.15.26:0/167804909 >> 172.21.15.26:0/167804909 conn(0x7f08e809e380 msgr2=0x7f08e80d4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.807+0000 7f08e7200640 1 -- 172.21.15.26:0/167804909 shutdown_connections 2024-04-26T21:09:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.807+0000 7f08e7200640 1 -- 172.21.15.26:0/167804909 wait complete. 2024-04-26T21:09:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.809+0000 7f08e7200640 1 Processor -- start 2024-04-26T21:09:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.809+0000 7f08e7200640 1 -- start start 2024-04-26T21:09:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.810+0000 7f08e7200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08cc0039b0 0x7f08e8160200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.810+0000 7f08e7200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08e81e9820 0x7f08e814c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.810+0000 7f08e7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e80d0b10 0x7f08e80d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.810+0000 7f08e7200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f08e815f670 con 0x7f08e81e9820 2024-04-26T21:09:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.810+0000 7f08e7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08e80d3540 con 0x7f08e80d0b10 2024-04-26T21:09:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.810+0000 7f08e7200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f08e81632e0 con 0x7f08cc0039b0 2024-04-26T21:09:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.811+0000 7f08e5400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08e81e9820 0x7f08e814c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.811+0000 7f08e5400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08e81e9820 0x7f08e814c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46896/0 (socket says 172.21.15.26:46896) 2024-04-26T21:09:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.811+0000 7f08e5400640 1 -- 172.21.15.26:0/1615721001 learned_addr learned my addr 172.21.15.26:0/1615721001 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.811+0000 7f08e5e00640 1 --2- 172.21.15.26:0/1615721001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08cc0039b0 0x7f08e8160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.811+0000 7f08e6800640 1 --2- 172.21.15.26:0/1615721001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e80d0b10 0x7f08e80d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.811+0000 7f08e5400640 1 -- 172.21.15.26:0/1615721001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08cc0039b0 msgr2=0x7f08e8160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.811+0000 7f08e5400640 1 --2- 172.21.15.26:0/1615721001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08cc0039b0 0x7f08e8160200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.811+0000 7f08e5400640 1 -- 172.21.15.26:0/1615721001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e80d0b10 msgr2=0x7f08e80d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.811+0000 7f08e5400640 1 --2- 172.21.15.26:0/1615721001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e80d0b10 0x7f08e80d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.811+0000 7f08e5400640 1 -- 172.21.15.26:0/1615721001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e81608e0 con 0x7f08e81e9820 2024-04-26T21:09:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.811+0000 7f08e6800640 1 --2- 172.21.15.26:0/1615721001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e80d0b10 0x7f08e80d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.811+0000 7f08e5400640 1 --2- 172.21.15.26:0/1615721001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08e81e9820 0x7f08e814c370 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f08d0011540 tx=0x7f08d0011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.812+0000 7f08de000640 1 -- 172.21.15.26:0/1615721001 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0015870 con 0x7f08e81e9820 2024-04-26T21:09:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.812+0000 7f08de000640 1 -- 172.21.15.26:0/1615721001 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d0015a10 con 0x7f08e81e9820 2024-04-26T21:09:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.812+0000 7f08e7200640 1 -- 172.21.15.26:0/1615721001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e8160bc0 con 0x7f08e81e9820 2024-04-26T21:09:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.812+0000 7f08de000640 1 -- 172.21.15.26:0/1615721001 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0015dc0 con 0x7f08e81e9820 2024-04-26T21:09:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.812+0000 7f08e7200640 1 -- 172.21.15.26:0/1615721001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e8161180 con 0x7f08e81e9820 2024-04-26T21:09:18.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.813+0000 7f08e7200640 1 -- 172.21.15.26:0/1615721001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08e806a700 con 0x7f08e81e9820 2024-04-26T21:09:18.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.814+0000 7f08de000640 1 -- 172.21.15.26:0/1615721001 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f08d000f460 con 0x7f08e81e9820 2024-04-26T21:09:18.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.815+0000 7f08de000640 1 --2- 172.21.15.26:0/1615721001 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f08c406c8f0 0x7f08c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:18.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.815+0000 7f08de000640 1 -- 172.21.15.26:0/1615721001 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f08d008c340 con 0x7f08e81e9820 2024-04-26T21:09:18.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.815+0000 7f08e5e00640 1 --2- 172.21.15.26:0/1615721001 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f08c406c8f0 0x7f08c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:18.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.816+0000 7f08e5e00640 1 --2- 172.21.15.26:0/1615721001 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f08c406c8f0 0x7f08c406edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f08d8033000 tx=0x7f08d8009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:18.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:18.820+0000 7f08de000640 1 -- 172.21.15.26:0/1615721001 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f08d0019030 con 0x7f08e81e9820 2024-04-26T21:09:19.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.025+0000 7f08e7200640 1 -- 172.21.15.26:0/1615721001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f08e80b76e0 con 0x7f08e81e9820 2024-04-26T21:09:19.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.026+0000 7f08de000640 1 -- 172.21.15.26:0/1615721001 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f08d005a770 con 0x7f08e81e9820 2024-04-26T21:09:19.028 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-26T21:09:19.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.029+0000 7f08e7200640 1 -- 172.21.15.26:0/1615721001 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f08c406c8f0 msgr2=0x7f08c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:19.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.029+0000 7f08e7200640 1 --2- 172.21.15.26:0/1615721001 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f08c406c8f0 0x7f08c406edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f08d8033000 tx=0x7f08d8009030 comp rx=0 tx=0).stop 2024-04-26T21:09:19.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.029+0000 7f08e7200640 1 -- 172.21.15.26:0/1615721001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08e81e9820 msgr2=0x7f08e814c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:19.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.029+0000 7f08e7200640 1 --2- 172.21.15.26:0/1615721001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08e81e9820 0x7f08e814c370 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f08d0011540 tx=0x7f08d0011a00 comp rx=0 tx=0).stop 2024-04-26T21:09:19.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.029+0000 7f08e7200640 1 -- 172.21.15.26:0/1615721001 shutdown_connections 2024-04-26T21:09:19.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.029+0000 7f08e7200640 1 --2- 172.21.15.26:0/1615721001 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f08c406c8f0 0x7f08c406edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.029+0000 7f08e7200640 1 --2- 172.21.15.26:0/1615721001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08e80d0b10 0x7f08e80d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.029+0000 7f08e7200640 1 --2- 172.21.15.26:0/1615721001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08e81e9820 0x7f08e814c370 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.029+0000 7f08e7200640 1 --2- 172.21.15.26:0/1615721001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08cc0039b0 0x7f08e8160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.029+0000 7f08e7200640 1 -- 172.21.15.26:0/1615721001 >> 172.21.15.26:0/1615721001 conn(0x7f08e809e380 msgr2=0x7f08e80b8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:19.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.029+0000 7f08e7200640 1 -- 172.21.15.26:0/1615721001 shutdown_connections 2024-04-26T21:09:19.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.029+0000 7f08e7200640 1 -- 172.21.15.26:0/1615721001 wait complete. 2024-04-26T21:09:19.050 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:09:19.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.265+0000 7fa4de400640 1 Processor -- start 2024-04-26T21:09:19.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.265+0000 7fa4de400640 1 -- start start 2024-04-26T21:09:19.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.267+0000 7fa4de400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4d80b5d30 0x7fa4d80b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:19.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.267+0000 7fa4de400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa4d806f8c0 0x7fa4d8166c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:19.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.267+0000 7fa4de400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4d8146c00 con 0x7fa4d80b42c0 2024-04-26T21:09:19.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.267+0000 7fa4de400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa4d8073700 con 0x7fa4d806f8c0 2024-04-26T21:09:19.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.267+0000 7fa4de400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa4d806e420 con 0x7fa4d80b5d30 2024-04-26T21:09:19.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.268+0000 7fa4d7400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4d80b5d30 0x7fa4d80b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:19.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.268+0000 7fa4d7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa4d806f8c0 0x7fa4d8166c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:19.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.268+0000 7fa4cea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa4d80b42c0 0x7fa4d80b6110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.26:55996/0 (socket says 172.21.15.26:55996) 2024-04-26T21:09:19.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.268+0000 7fa4cea00640 1 -- 172.21.15.26:0/1534719493 learned_addr learned my addr 172.21.15.26:0/1534719493 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:19.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.269+0000 7fa4d7e00640 1 -- 172.21.15.26:0/1534719493 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4d80b5d30 msgr2=0x7fa4d80b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:19.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.269+0000 7fa4d7e00640 1 --2- 172.21.15.26:0/1534719493 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4d80b5d30 0x7fa4d80b3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.269+0000 7fa4d7e00640 1 -- 172.21.15.26:0/1534719493 >> v1:172.21.15.86:6789/0 conn(0x7fa4d80b42c0 legacy=0x7fa4d80b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:19.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.269+0000 7fa4d7e00640 1 -- 172.21.15.26:0/1534719493 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d8072e60 con 0x7fa4d806f8c0 2024-04-26T21:09:19.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.270+0000 7fa4d7e00640 1 --2- 172.21.15.26:0/1534719493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4d806f8c0 0x7fa4d8166c90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009d90 tx=0x7fa4d002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f793852f0dcf5cac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:19.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.271+0000 7fa4d6a00640 1 -- 172.21.15.26:0/1534719493 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0007740 con 0x7fa4d806f8c0 2024-04-26T21:09:19.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.271+0000 7fa4d6a00640 1 -- 172.21.15.26:0/1534719493 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4d00078e0 con 0x7fa4d806f8c0 2024-04-26T21:09:19.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.271+0000 7fa4d6a00640 1 -- 172.21.15.26:0/1534719493 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0007cb0 con 0x7fa4d806f8c0 2024-04-26T21:09:19.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.271+0000 7fa4de400640 1 -- 172.21.15.26:0/1534719493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4d806f8c0 msgr2=0x7fa4d8166c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:19.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.271+0000 7fa4de400640 1 --2- 172.21.15.26:0/1534719493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4d806f8c0 0x7fa4d8166c90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009d90 tx=0x7fa4d002a660 comp rx=0 tx=0).stop 2024-04-26T21:09:19.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.271+0000 7fa4de400640 1 -- 172.21.15.26:0/1534719493 shutdown_connections 2024-04-26T21:09:19.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.271+0000 7fa4de400640 1 --2- 172.21.15.26:0/1534719493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4d806f8c0 0x7fa4d8166c90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.271+0000 7fa4de400640 1 --2- 172.21.15.26:0/1534719493 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4d80b5d30 0x7fa4d80b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.271+0000 7fa4de400640 1 -- 172.21.15.26:0/1534719493 >> 172.21.15.26:0/1534719493 conn(0x7fa4d80b2a80 msgr2=0x7fa4d80b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:19.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.272+0000 7fa4de400640 1 -- 172.21.15.26:0/1534719493 shutdown_connections 2024-04-26T21:09:19.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.272+0000 7fa4de400640 1 -- 172.21.15.26:0/1534719493 wait complete. 2024-04-26T21:09:19.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.274+0000 7fa4de400640 1 Processor -- start 2024-04-26T21:09:19.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.274+0000 7fa4de400640 1 -- start start 2024-04-26T21:09:19.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.275+0000 7fa4de400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4d806f8c0 0x7fa4d8142170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:19.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.275+0000 7fa4de400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4d80b42c0 0x7fa4d8144ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:19.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.275+0000 7fa4de400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d80b5d30 0x7fa4d81db150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:19.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.275+0000 7fa4de400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d80b6820 con 0x7fa4d80b5d30 2024-04-26T21:09:19.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.275+0000 7fa4de400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d8073420 con 0x7fa4d806f8c0 2024-04-26T21:09:19.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.275+0000 7fa4de400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa4d8072a80 con 0x7fa4d80b42c0 2024-04-26T21:09:19.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.275+0000 7fa4d7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d80b5d30 0x7fa4d81db150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:19.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.275+0000 7fa4d7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d80b5d30 0x7fa4d81db150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50890/0 (socket says 172.21.15.26:50890) 2024-04-26T21:09:19.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.275+0000 7fa4d7e00640 1 -- 172.21.15.26:0/3598333269 learned_addr learned my addr 172.21.15.26:0/3598333269 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:19.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.275+0000 7fa4d7400640 1 --2- 172.21.15.26:0/3598333269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4d806f8c0 0x7fa4d8142170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:19.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.275+0000 7fa4cea00640 1 --2- 172.21.15.26:0/3598333269 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4d80b42c0 0x7fa4d8144ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:19.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.276+0000 7fa4d7e00640 1 -- 172.21.15.26:0/3598333269 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4d80b42c0 msgr2=0x7fa4d8144ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:19.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.276+0000 7fa4d7e00640 1 --2- 172.21.15.26:0/3598333269 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4d80b42c0 0x7fa4d8144ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.276+0000 7fa4d7e00640 1 -- 172.21.15.26:0/3598333269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4d806f8c0 msgr2=0x7fa4d8142170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:19.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.276+0000 7fa4d7e00640 1 --2- 172.21.15.26:0/3598333269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4d806f8c0 0x7fa4d8142170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.276+0000 7fa4d7e00640 1 -- 172.21.15.26:0/3598333269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d8145550 con 0x7fa4d80b5d30 2024-04-26T21:09:19.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.276+0000 7fa4d7e00640 1 --2- 172.21.15.26:0/3598333269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d80b5d30 0x7fa4d81db150 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009eb0 tx=0x7fa4d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:19.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.276+0000 7fa4cfe00640 1 -- 172.21.15.26:0/3598333269 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0040070 con 0x7fa4d80b5d30 2024-04-26T21:09:19.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.276+0000 7fa4cfe00640 1 -- 172.21.15.26:0/3598333269 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4d0031bc0 con 0x7fa4d80b5d30 2024-04-26T21:09:19.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.276+0000 7fa4cfe00640 1 -- 172.21.15.26:0/3598333269 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0009450 con 0x7fa4d80b5d30 2024-04-26T21:09:19.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.276+0000 7fa4de400640 1 -- 172.21.15.26:0/3598333269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d8145830 con 0x7fa4d80b5d30 2024-04-26T21:09:19.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.277+0000 7fa4de400640 1 -- 172.21.15.26:0/3598333269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d81f0c30 con 0x7fa4d80b5d30 2024-04-26T21:09:19.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.278+0000 7fa4de400640 1 -- 172.21.15.26:0/3598333269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4d806a700 con 0x7fa4d80b5d30 2024-04-26T21:09:19.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.279+0000 7fa4cfe00640 1 -- 172.21.15.26:0/3598333269 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4d0039040 con 0x7fa4d80b5d30 2024-04-26T21:09:19.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.280+0000 7fa4cfe00640 1 --2- 172.21.15.26:0/3598333269 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa4b406c8f0 0x7fa4b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:19.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.280+0000 7fa4cfe00640 1 -- 172.21.15.26:0/3598333269 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa4d00ad450 con 0x7fa4d80b5d30 2024-04-26T21:09:19.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.280+0000 7fa4d7400640 1 --2- 172.21.15.26:0/3598333269 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa4b406c8f0 0x7fa4b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:19.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.281+0000 7fa4d7400640 1 --2- 172.21.15.26:0/3598333269 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa4b406c8f0 0x7fa4b406edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa4d81db100 tx=0x7fa4c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:19.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.285+0000 7fa4cfe00640 1 -- 172.21.15.26:0/3598333269 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa4d007b900 con 0x7fa4d80b5d30 2024-04-26T21:09:19.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.492+0000 7fa4de400640 1 -- 172.21.15.26:0/3598333269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa4d8145a00 con 0x7fa4d80b5d30 2024-04-26T21:09:19.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.493+0000 7fa4cfe00640 1 -- 172.21.15.26:0/3598333269 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fa4d007b200 con 0x7fa4d80b5d30 2024-04-26T21:09:19.494 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-26T21:09:19.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.496+0000 7fa4de400640 1 -- 172.21.15.26:0/3598333269 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa4b406c8f0 msgr2=0x7fa4b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:19.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.496+0000 7fa4de400640 1 --2- 172.21.15.26:0/3598333269 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa4b406c8f0 0x7fa4b406edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa4d81db100 tx=0x7fa4c8008040 comp rx=0 tx=0).stop 2024-04-26T21:09:19.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.496+0000 7fa4de400640 1 -- 172.21.15.26:0/3598333269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d80b5d30 msgr2=0x7fa4d81db150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:19.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.496+0000 7fa4de400640 1 --2- 172.21.15.26:0/3598333269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d80b5d30 0x7fa4d81db150 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009eb0 tx=0x7fa4d0004a40 comp rx=0 tx=0).stop 2024-04-26T21:09:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.496+0000 7fa4de400640 1 -- 172.21.15.26:0/3598333269 shutdown_connections 2024-04-26T21:09:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.496+0000 7fa4de400640 1 --2- 172.21.15.26:0/3598333269 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa4b406c8f0 0x7fa4b406edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.496+0000 7fa4de400640 1 --2- 172.21.15.26:0/3598333269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d80b5d30 0x7fa4d81db150 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.496+0000 7fa4de400640 1 --2- 172.21.15.26:0/3598333269 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4d80b42c0 0x7fa4d8144ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.496+0000 7fa4de400640 1 --2- 172.21.15.26:0/3598333269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4d806f8c0 0x7fa4d8142170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.496+0000 7fa4de400640 1 -- 172.21.15.26:0/3598333269 >> 172.21.15.26:0/3598333269 conn(0x7fa4d80b2a80 msgr2=0x7fa4d80cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.496+0000 7fa4de400640 1 -- 172.21.15.26:0/3598333269 shutdown_connections 2024-04-26T21:09:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.496+0000 7fa4de400640 1 -- 172.21.15.26:0/3598333269 wait complete. 2024-04-26T21:09:19.518 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:09:19.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.732+0000 7f34c9e00640 1 Processor -- start 2024-04-26T21:09:19.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.733+0000 7f34c9e00640 1 -- start start 2024-04-26T21:09:19.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.734+0000 7f34c9e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34c40b74d0 0x7f34c40d0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:19.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.734+0000 7f34c9e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34c40b7f90 con 0x7f34c40d3b90 2024-04-26T21:09:19.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.734+0000 7f34c9e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34c40b8270 con 0x7f34c40d0ab0 2024-04-26T21:09:19.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.734+0000 7f34c9e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f34c40b8550 con 0x7f34c40b74d0 2024-04-26T21:09:19.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.735+0000 7f34c9400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f34c40d3b90 0x7f34c415eb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54552/0 (socket says 172.21.15.26:54552) 2024-04-26T21:09:19.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.735+0000 7f34c9400640 1 -- 172.21.15.26:0/823565346 learned_addr learned my addr 172.21.15.26:0/823565346 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:19.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.735+0000 7f34c8a00640 1 --2- 172.21.15.26:0/823565346 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34c40b74d0 0x7f34c40d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:19.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.735+0000 7f34c3400640 1 -- 172.21.15.26:0/823565346 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3009337260 0 0) 0x7f34c40b7f90 con 0x7f34c40d3b90 2024-04-26T21:09:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.735+0000 7f34c3400640 1 -- 172.21.15.26:0/823565346 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34a8003520 con 0x7f34c40d3b90 2024-04-26T21:09:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.736+0000 7f34c3400640 1 -- 172.21.15.26:0/823565346 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2861902041 0 0) 0x7f34a8003520 con 0x7f34c40d3b90 2024-04-26T21:09:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.736+0000 7f34c3400640 1 -- 172.21.15.26:0/823565346 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34c40b74d0 msgr2=0x7f34c40d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.736+0000 7f34c3400640 1 --2- 172.21.15.26:0/823565346 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34c40b74d0 0x7f34c40d0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.736+0000 7f34c3400640 1 -- 172.21.15.26:0/823565346 >> v1:172.21.15.86:6789/0 conn(0x7f34c40d0ab0 legacy=0x7f34c40d3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.736+0000 7f34c3400640 1 -- 172.21.15.26:0/823565346 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c40b8830 con 0x7f34c40d3b90 2024-04-26T21:09:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.736+0000 7f34c3400640 1 -- 172.21.15.26:0/823565346 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f34bc001670 con 0x7f34c40d3b90 2024-04-26T21:09:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.736+0000 7f34c3400640 1 -- 172.21.15.26:0/823565346 >> v1:172.21.15.26:6789/0 conn(0x7f34c40d3b90 legacy=0x7f34c415eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.736+0000 7f34c3400640 1 --2- 172.21.15.26:0/823565346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a80039e0 0x7f34a8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.736+0000 7f34c3400640 1 --2- 172.21.15.26:0/823565346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a8004390 0x7f34a8004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.736+0000 7f34c3400640 1 --2- 172.21.15.26:0/823565346 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34a8005060 0x7f34a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.736+0000 7f34c3400640 1 -- 172.21.15.26:0/823565346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f34c40b8550 con 0x7f34a80039e0 2024-04-26T21:09:19.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.736+0000 7f34c3400640 1 -- 172.21.15.26:0/823565346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34a800bf30 con 0x7f34a8004390 2024-04-26T21:09:19.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.736+0000 7f34c3400640 1 -- 172.21.15.26:0/823565346 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f34a800c130 con 0x7f34a8005060 2024-04-26T21:09:19.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.736+0000 7f34c8a00640 1 --2- 172.21.15.26:0/823565346 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34c40b74d0 0x7f34c40d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:09:19.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.736+0000 7f34c8a00640 1 --2- 172.21.15.26:0/823565346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a80039e0 0x7f34a8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:19.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.737+0000 7f34c9400640 1 --2- 172.21.15.26:0/823565346 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34a8005060 0x7f34a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:19.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.737+0000 7f34c3e00640 1 --2- 172.21.15.26:0/823565346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a8004390 0x7f34a8004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:19.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.737+0000 7f34c8a00640 1 -- 172.21.15.26:0/823565346 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34a8005060 msgr2=0x7f34a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:19.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.737+0000 7f34c8a00640 1 --2- 172.21.15.26:0/823565346 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34a8005060 0x7f34a800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.737+0000 7f34c8a00640 1 -- 172.21.15.26:0/823565346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a8004390 msgr2=0x7f34a8004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:19.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.737+0000 7f34c8a00640 1 --2- 172.21.15.26:0/823565346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a8004390 0x7f34a8004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.737+0000 7f34c8a00640 1 -- 172.21.15.26:0/823565346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a8003520 con 0x7f34a80039e0 2024-04-26T21:09:19.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.738+0000 7f34c8a00640 1 --2- 172.21.15.26:0/823565346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a80039e0 0x7f34a8003dc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f34c408d210 tx=0x7f34a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130f3b104da0d979 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:19.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.738+0000 7f34c3400640 1 -- 172.21.15.26:0/823565346 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4036070 con 0x7f34a80039e0 2024-04-26T21:09:19.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.738+0000 7f34c3400640 1 -- 172.21.15.26:0/823565346 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34b4003c50 con 0x7f34a80039e0 2024-04-26T21:09:19.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.738+0000 7f34c3400640 1 -- 172.21.15.26:0/823565346 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4009d40 con 0x7f34a80039e0 2024-04-26T21:09:19.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.739+0000 7f34c9e00640 1 -- 172.21.15.26:0/823565346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a80039e0 msgr2=0x7f34a8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.739+0000 7f34c9e00640 1 --2- 172.21.15.26:0/823565346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34a80039e0 0x7f34a8003dc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f34c408d210 tx=0x7f34a8005830 comp rx=0 tx=0).stop 2024-04-26T21:09:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.739+0000 7f34c9400640 1 -- 172.21.15.26:0/823565346 reap_dead start 2024-04-26T21:09:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.739+0000 7f34c9e00640 1 -- 172.21.15.26:0/823565346 shutdown_connections 2024-04-26T21:09:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.739+0000 7f34c9e00640 1 -- 172.21.15.26:0/823565346 >> 172.21.15.26:0/823565346 conn(0x7f34c40d4f30 msgr2=0x7f34c40d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.739+0000 7f34c9e00640 1 -- 172.21.15.26:0/823565346 shutdown_connections 2024-04-26T21:09:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.739+0000 7f34c9e00640 1 -- 172.21.15.26:0/823565346 wait complete. 2024-04-26T21:09:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.742+0000 7f34c9e00640 1 Processor -- start 2024-04-26T21:09:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.742+0000 7f34c9e00640 1 -- start start 2024-04-26T21:09:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.743+0000 7f34c9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a80039e0 0x7f34c40a27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.743+0000 7f34c9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c40a1420 0x7f34c409fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.743+0000 7f34c9e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34c4152270 0x7f34c41e90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.743+0000 7f34c9e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f34c40ba040 con 0x7f34c40a1420 2024-04-26T21:09:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.743+0000 7f34c9e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34c40b78b0 con 0x7f34a80039e0 2024-04-26T21:09:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.743+0000 7f34c9e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f34c40ba1c0 con 0x7f34c4152270 2024-04-26T21:09:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.744+0000 7f34c3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c40a1420 0x7f34c409fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.744+0000 7f34c3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c40a1420 0x7f34c409fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50912/0 (socket says 172.21.15.26:50912) 2024-04-26T21:09:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.744+0000 7f34c3e00640 1 -- 172.21.15.26:0/3988711856 learned_addr learned my addr 172.21.15.26:0/3988711856 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.744+0000 7f34c8a00640 1 --2- 172.21.15.26:0/3988711856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a80039e0 0x7f34c40a27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.744+0000 7f34c9400640 1 --2- 172.21.15.26:0/3988711856 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34c4152270 0x7f34c41e90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:19.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.744+0000 7f34c3e00640 1 -- 172.21.15.26:0/3988711856 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34c4152270 msgr2=0x7f34c41e90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:19.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.744+0000 7f34c3e00640 1 --2- 172.21.15.26:0/3988711856 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34c4152270 0x7f34c41e90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.744+0000 7f34c3e00640 1 -- 172.21.15.26:0/3988711856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a80039e0 msgr2=0x7f34c40a27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:19.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.744+0000 7f34c3e00640 1 --2- 172.21.15.26:0/3988711856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a80039e0 0x7f34c40a27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.744+0000 7f34c3e00640 1 -- 172.21.15.26:0/3988711856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c40a0380 con 0x7f34c40a1420 2024-04-26T21:09:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.744+0000 7f34c8a00640 1 --2- 172.21.15.26:0/3988711856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a80039e0 0x7f34c40a27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:09:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.745+0000 7f34c3e00640 1 --2- 172.21.15.26:0/3988711856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c40a1420 0x7f34c409fc80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00e800 tx=0x7f34ac00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.745+0000 7f34c0c00640 1 -- 172.21.15.26:0/3988711856 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac0050d0 con 0x7f34c40a1420 2024-04-26T21:09:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.745+0000 7f34c0c00640 1 -- 172.21.15.26:0/3988711856 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34ac018070 con 0x7f34c40a1420 2024-04-26T21:09:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.745+0000 7f34c0c00640 1 -- 172.21.15.26:0/3988711856 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac0054b0 con 0x7f34c40a1420 2024-04-26T21:09:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.745+0000 7f34c9e00640 1 -- 172.21.15.26:0/3988711856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34c40a0660 con 0x7f34c40a1420 2024-04-26T21:09:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.745+0000 7f34c9e00640 1 -- 172.21.15.26:0/3988711856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34c40a0c20 con 0x7f34c40a1420 2024-04-26T21:09:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.746+0000 7f34c9e00640 1 -- 172.21.15.26:0/3988711856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34c40506c0 con 0x7f34c40a1420 2024-04-26T21:09:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.747+0000 7f34c0c00640 1 -- 172.21.15.26:0/3988711856 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f34ac00c6c0 con 0x7f34c40a1420 2024-04-26T21:09:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.749+0000 7f34c0c00640 1 --2- 172.21.15.26:0/3988711856 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f34a006c8f0 0x7f34a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.749+0000 7f34c0c00640 1 -- 172.21.15.26:0/3988711856 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f34ac08b820 con 0x7f34c40a1420 2024-04-26T21:09:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.749+0000 7f34c8a00640 1 --2- 172.21.15.26:0/3988711856 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f34a006c8f0 0x7f34a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.750+0000 7f34c8a00640 1 --2- 172.21.15.26:0/3988711856 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f34a006c8f0 0x7f34a006edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f34b4033000 tx=0x7f34b402f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.753+0000 7f34c0c00640 1 -- 172.21.15.26:0/3988711856 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f34ac059ce0 con 0x7f34c40a1420 2024-04-26T21:09:19.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.957+0000 7f34c9e00640 1 -- 172.21.15.26:0/3988711856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f34c40d3260 con 0x7f34c40a1420 2024-04-26T21:09:19.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.957+0000 7f34c0c00640 1 -- 172.21.15.26:0/3988711856 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f34ac0595e0 con 0x7f34c40a1420 2024-04-26T21:09:19.959 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-26T21:09:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.960+0000 7f34c9e00640 1 -- 172.21.15.26:0/3988711856 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f34a006c8f0 msgr2=0x7f34a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.960+0000 7f34c9e00640 1 --2- 172.21.15.26:0/3988711856 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f34a006c8f0 0x7f34a006edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f34b4033000 tx=0x7f34b402f000 comp rx=0 tx=0).stop 2024-04-26T21:09:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.960+0000 7f34c9e00640 1 -- 172.21.15.26:0/3988711856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c40a1420 msgr2=0x7f34c409fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.960+0000 7f34c9e00640 1 --2- 172.21.15.26:0/3988711856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c40a1420 0x7f34c409fc80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00e800 tx=0x7f34ac00ecc0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.960+0000 7f34c9e00640 1 -- 172.21.15.26:0/3988711856 shutdown_connections 2024-04-26T21:09:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.960+0000 7f34c9e00640 1 --2- 172.21.15.26:0/3988711856 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f34a006c8f0 0x7f34a006edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.960+0000 7f34c9e00640 1 --2- 172.21.15.26:0/3988711856 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34c4152270 0x7f34c41e90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.960+0000 7f34c9e00640 1 --2- 172.21.15.26:0/3988711856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c40a1420 0x7f34c409fc80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.960+0000 7f34c9e00640 1 --2- 172.21.15.26:0/3988711856 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34a80039e0 0x7f34c40a27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.960+0000 7f34c9e00640 1 -- 172.21.15.26:0/3988711856 >> 172.21.15.26:0/3988711856 conn(0x7f34c40d4f30 msgr2=0x7f34c415feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.961+0000 7f34c9e00640 1 -- 172.21.15.26:0/3988711856 shutdown_connections 2024-04-26T21:09:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:19.961+0000 7f34c9e00640 1 -- 172.21.15.26:0/3988711856 wait complete. 2024-04-26T21:09:19.983 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace' 2024-04-26T21:09:20.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.198+0000 7f6340400640 1 Processor -- start 2024-04-26T21:09:20.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.198+0000 7f6340400640 1 -- start start 2024-04-26T21:09:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.199+0000 7f6340400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f633806f110 0x7f63380b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.199+0000 7f6340400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f63380b4fe0 0x7f63380b6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.199+0000 7f6340400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63380d2db0 con 0x7f633815f450 2024-04-26T21:09:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.199+0000 7f6340400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f63380d2ad0 con 0x7f63380b4fe0 2024-04-26T21:09:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.199+0000 7f6340400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f63380d0c60 con 0x7f633806f110 2024-04-26T21:09:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.200+0000 7f633e400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f63380b4fe0 0x7f63380b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.200+0000 7f633e400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f63380b4fe0 0x7f63380b6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50928/0 (socket says 172.21.15.26:50928) 2024-04-26T21:09:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.200+0000 7f633e400640 1 -- 172.21.15.26:0/579950033 learned_addr learned my addr 172.21.15.26:0/579950033 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.200+0000 7f633d000640 1 --2- 172.21.15.26:0/579950033 >> v2:172.21.15.86:3300/0 conn(0x7f633806f110 0x7f63380b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.200+0000 7f633e400640 1 -- 172.21.15.26:0/579950033 >> v2:172.21.15.86:3300/0 conn(0x7f633806f110 msgr2=0x7f63380b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.200+0000 7f633e400640 1 --2- 172.21.15.26:0/579950033 >> v2:172.21.15.86:3300/0 conn(0x7f633806f110 0x7f63380b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:20.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.200+0000 7f633e400640 1 -- 172.21.15.26:0/579950033 >> v1:172.21.15.86:6789/0 conn(0x7f633815f450 legacy=0x7f6338163880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:20.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.200+0000 7f633e400640 1 -- 172.21.15.26:0/579950033 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63380d0f40 con 0x7f63380b4fe0 2024-04-26T21:09:20.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.200+0000 7f633d000640 1 --2- 172.21.15.26:0/579950033 >> v2:172.21.15.86:3300/0 conn(0x7f633806f110 0x7f63380b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:20.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.201+0000 7f633e400640 1 --2- 172.21.15.26:0/579950033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63380b4fe0 0x7f63380b6140 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f633806d260 tx=0x7f633002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674ced7a39c5911 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:20.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.202+0000 7f6337e00640 1 -- 172.21.15.26:0/579950033 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6330036070 con 0x7f63380b4fe0 2024-04-26T21:09:20.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.202+0000 7f6337e00640 1 -- 172.21.15.26:0/579950033 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6330031600 con 0x7f63380b4fe0 2024-04-26T21:09:20.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.202+0000 7f6337e00640 1 -- 172.21.15.26:0/579950033 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63300319d0 con 0x7f63380b4fe0 2024-04-26T21:09:20.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.202+0000 7f6340400640 1 -- 172.21.15.26:0/579950033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63380b4fe0 msgr2=0x7f63380b6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:20.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.202+0000 7f6340400640 1 --2- 172.21.15.26:0/579950033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63380b4fe0 0x7f63380b6140 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f633806d260 tx=0x7f633002a660 comp rx=0 tx=0).stop 2024-04-26T21:09:20.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.202+0000 7f6340400640 1 -- 172.21.15.26:0/579950033 shutdown_connections 2024-04-26T21:09:20.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.202+0000 7f6340400640 1 --2- 172.21.15.26:0/579950033 >> v2:172.21.15.86:3300/0 conn(0x7f633806f110 0x7f63380b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:20.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.202+0000 7f6340400640 1 --2- 172.21.15.26:0/579950033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63380b4fe0 0x7f63380b6140 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:20.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.202+0000 7f6340400640 1 -- 172.21.15.26:0/579950033 >> 172.21.15.26:0/579950033 conn(0x7f633806c240 msgr2=0x7f633806c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:20.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.202+0000 7f6340400640 1 -- 172.21.15.26:0/579950033 shutdown_connections 2024-04-26T21:09:20.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.203+0000 7f6340400640 1 -- 172.21.15.26:0/579950033 wait complete. 2024-04-26T21:09:20.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.204+0000 7f6340400640 1 Processor -- start 2024-04-26T21:09:20.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.204+0000 7f6340400640 1 -- start start 2024-04-26T21:09:20.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.205+0000 7f6340400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f633806f110 0x7f63380c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:20.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.205+0000 7f6340400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63380b4fe0 0x7f63380cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:20.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.205+0000 7f6340400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f633815f450 0x7f6338152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:20.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.205+0000 7f6340400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f63380d28b0 con 0x7f63380b4fe0 2024-04-26T21:09:20.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.205+0000 7f6340400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63380d2730 con 0x7f633815f450 2024-04-26T21:09:20.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.205+0000 7f6340400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f63380b6680 con 0x7f633806f110 2024-04-26T21:09:20.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.206+0000 7f633d000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63380b4fe0 0x7f63380cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:20.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.206+0000 7f633d000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63380b4fe0 0x7f63380cbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50932/0 (socket says 172.21.15.26:50932) 2024-04-26T21:09:20.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.206+0000 7f633d000640 1 -- 172.21.15.26:0/2213571410 learned_addr learned my addr 172.21.15.26:0/2213571410 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:20.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.206+0000 7f633da00640 1 --2- 172.21.15.26:0/2213571410 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f633806f110 0x7f63380c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:20.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.206+0000 7f633e400640 1 --2- 172.21.15.26:0/2213571410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f633815f450 0x7f6338152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:20.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.206+0000 7f633d000640 1 -- 172.21.15.26:0/2213571410 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f633806f110 msgr2=0x7f63380c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:20.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.206+0000 7f633d000640 1 --2- 172.21.15.26:0/2213571410 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f633806f110 0x7f63380c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:20.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.206+0000 7f633d000640 1 -- 172.21.15.26:0/2213571410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f633815f450 msgr2=0x7f6338152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:20.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.206+0000 7f633d000640 1 --2- 172.21.15.26:0/2213571410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f633815f450 0x7f6338152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:20.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.206+0000 7f633d000640 1 -- 172.21.15.26:0/2213571410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63380caeb0 con 0x7f63380b4fe0 2024-04-26T21:09:20.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.207+0000 7f633d000640 1 --2- 172.21.15.26:0/2213571410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63380b4fe0 0x7f63380cbe10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f632400c7f0 tx=0x7f632400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:20.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.207+0000 7f6335600640 1 -- 172.21.15.26:0/2213571410 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324015070 con 0x7f63380b4fe0 2024-04-26T21:09:20.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.207+0000 7f6335600640 1 -- 172.21.15.26:0/2213571410 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6324007a30 con 0x7f63380b4fe0 2024-04-26T21:09:20.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.207+0000 7f6335600640 1 -- 172.21.15.26:0/2213571410 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324007e00 con 0x7f63380b4fe0 2024-04-26T21:09:20.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.207+0000 7f6340400640 1 -- 172.21.15.26:0/2213571410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63380ca0f0 con 0x7f63380b4fe0 2024-04-26T21:09:20.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.207+0000 7f6340400640 1 -- 172.21.15.26:0/2213571410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63380ca6b0 con 0x7f63380b4fe0 2024-04-26T21:09:20.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.208+0000 7f6340400640 1 -- 172.21.15.26:0/2213571410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f633806a700 con 0x7f63380b4fe0 2024-04-26T21:09:20.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.209+0000 7f6335600640 1 -- 172.21.15.26:0/2213571410 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6324018070 con 0x7f63380b4fe0 2024-04-26T21:09:20.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.210+0000 7f6335600640 1 --2- 172.21.15.26:0/2213571410 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f631806c8f0 0x7f631806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:20.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.211+0000 7f6335600640 1 -- 172.21.15.26:0/2213571410 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f632408bbb0 con 0x7f63380b4fe0 2024-04-26T21:09:20.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.211+0000 7f633da00640 1 --2- 172.21.15.26:0/2213571410 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f631806c8f0 0x7f631806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:20.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.211+0000 7f633da00640 1 --2- 172.21.15.26:0/2213571410 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f631806c8f0 0x7f631806edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f632c006fd0 tx=0x7f632c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:20.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.215+0000 7f6335600640 1 -- 172.21.15.26:0/2213571410 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6324059f40 con 0x7f63380b4fe0 2024-04-26T21:09:20.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.493+0000 7f6340400640 1 -- 172.21.15.26:0/2213571410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace"]} v 0) -- 0x7f63380c6410 con 0x7f63380b4fe0 2024-04-26T21:09:20.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.543+0000 7f6335600640 1 -- 172.21.15.26:0/2213571410 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace"]}]=0 v58) ==== 138+0+0 (secure 0 0 0) 0x7f6324059840 con 0x7f63380b4fe0 2024-04-26T21:09:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.546+0000 7f6340400640 1 -- 172.21.15.26:0/2213571410 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f631806c8f0 msgr2=0x7f631806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.546+0000 7f6340400640 1 --2- 172.21.15.26:0/2213571410 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f631806c8f0 0x7f631806edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f632c006fd0 tx=0x7f632c008040 comp rx=0 tx=0).stop 2024-04-26T21:09:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.546+0000 7f6340400640 1 -- 172.21.15.26:0/2213571410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63380b4fe0 msgr2=0x7f63380cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.546+0000 7f6340400640 1 --2- 172.21.15.26:0/2213571410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63380b4fe0 0x7f63380cbe10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f632400c7f0 tx=0x7f632400cbf0 comp rx=0 tx=0).stop 2024-04-26T21:09:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.546+0000 7f6340400640 1 -- 172.21.15.26:0/2213571410 shutdown_connections 2024-04-26T21:09:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.546+0000 7f6340400640 1 --2- 172.21.15.26:0/2213571410 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f631806c8f0 0x7f631806edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.546+0000 7f6340400640 1 --2- 172.21.15.26:0/2213571410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f633815f450 0x7f6338152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.546+0000 7f6340400640 1 --2- 172.21.15.26:0/2213571410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63380b4fe0 0x7f63380cbe10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.546+0000 7f6340400640 1 --2- 172.21.15.26:0/2213571410 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f633806f110 0x7f63380c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.546+0000 7f6340400640 1 -- 172.21.15.26:0/2213571410 >> 172.21.15.26:0/2213571410 conn(0x7f633806c240 msgr2=0x7f63380cd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.547+0000 7f6340400640 1 -- 172.21.15.26:0/2213571410 shutdown_connections 2024-04-26T21:09:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.547+0000 7f6340400640 1 -- 172.21.15.26:0/2213571410 wait complete. 2024-04-26T21:09:20.568 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T21:09:20.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.784+0000 7f1091e00640 1 Processor -- start 2024-04-26T21:09:20.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.784+0000 7f1091e00640 1 -- start start 2024-04-26T21:09:20.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.786+0000 7f1091e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f108c070c30 0x7f108c071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:20.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.786+0000 7f1091e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f108c0d2220 0x7f108c0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:20.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.786+0000 7f1091e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f108c071550 con 0x7f108c0d2aa0 2024-04-26T21:09:20.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.786+0000 7f1091e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f108c06d350 con 0x7f108c070c30 2024-04-26T21:09:20.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.786+0000 7f1091e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f108c0cdd10 con 0x7f108c0d2220 2024-04-26T21:09:20.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.786+0000 7f1090a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f108c070c30 0x7f108c071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:20.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.787+0000 7f1090a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f108c070c30 0x7f108c071010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50948/0 (socket says 172.21.15.26:50948) 2024-04-26T21:09:20.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.787+0000 7f1090a00640 1 -- 172.21.15.26:0/3770399951 learned_addr learned my addr 172.21.15.26:0/3770399951 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:20.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.787+0000 7f1091400640 1 --2- 172.21.15.26:0/3770399951 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f108c0d2220 0x7f108c0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:20.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.787+0000 7f1090a00640 1 -- 172.21.15.26:0/3770399951 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f108c0d2220 msgr2=0x7f108c0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:20.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.787+0000 7f1090a00640 1 --2- 172.21.15.26:0/3770399951 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f108c0d2220 0x7f108c0ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:20.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.787+0000 7f1090a00640 1 -- 172.21.15.26:0/3770399951 >> v1:172.21.15.26:6789/0 conn(0x7f108c0d2aa0 legacy=0x7f108c0ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:20.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.787+0000 7f1090a00640 1 -- 172.21.15.26:0/3770399951 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108c070560 con 0x7f108c070c30 2024-04-26T21:09:20.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.788+0000 7f1090a00640 1 --2- 172.21.15.26:0/3770399951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f108c070c30 0x7f108c071010 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1080009870 tx=0x7f108002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce85a8c398023757 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:20.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.788+0000 7f108b400640 1 -- 172.21.15.26:0/3770399951 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1080037070 con 0x7f108c070c30 2024-04-26T21:09:20.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.789+0000 7f108b400640 1 -- 172.21.15.26:0/3770399951 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1080032440 con 0x7f108c070c30 2024-04-26T21:09:20.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.789+0000 7f108b400640 1 -- 172.21.15.26:0/3770399951 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1080032810 con 0x7f108c070c30 2024-04-26T21:09:20.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.789+0000 7f1091e00640 1 -- 172.21.15.26:0/3770399951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f108c070c30 msgr2=0x7f108c071010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:20.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.789+0000 7f1091e00640 1 --2- 172.21.15.26:0/3770399951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f108c070c30 0x7f108c071010 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1080009870 tx=0x7f108002a130 comp rx=0 tx=0).stop 2024-04-26T21:09:20.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.789+0000 7f1091e00640 1 -- 172.21.15.26:0/3770399951 shutdown_connections 2024-04-26T21:09:20.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.789+0000 7f1091e00640 1 --2- 172.21.15.26:0/3770399951 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f108c0d2220 0x7f108c0ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:20.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.789+0000 7f1091e00640 1 --2- 172.21.15.26:0/3770399951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f108c070c30 0x7f108c071010 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:20.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.789+0000 7f1091e00640 1 -- 172.21.15.26:0/3770399951 >> 172.21.15.26:0/3770399951 conn(0x7f108c0d2ed0 msgr2=0x7f108c0d32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:20.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.789+0000 7f1091e00640 1 -- 172.21.15.26:0/3770399951 shutdown_connections 2024-04-26T21:09:20.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.789+0000 7f1091e00640 1 -- 172.21.15.26:0/3770399951 wait complete. 2024-04-26T21:09:20.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.791+0000 7f1091e00640 1 Processor -- start 2024-04-26T21:09:20.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.791+0000 7f1091e00640 1 -- start start 2024-04-26T21:09:20.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.792+0000 7f1091e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f108c070c30 0x7f108c1f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:20.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.792+0000 7f1091e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f108c0d2220 0x7f108c15e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:20.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.792+0000 7f1091e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f108c0d2aa0 0x7f108c15c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:20.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.792+0000 7f1091e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f108c06f210 con 0x7f108c0d2220 2024-04-26T21:09:20.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.792+0000 7f1091e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f108c0cd9a0 con 0x7f108c070c30 2024-04-26T21:09:20.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.792+0000 7f1091e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f108c0d0690 con 0x7f108c0d2aa0 2024-04-26T21:09:20.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.792+0000 7f108be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f108c0d2220 0x7f108c15e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:20.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.793+0000 7f108be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f108c0d2220 0x7f108c15e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50960/0 (socket says 172.21.15.26:50960) 2024-04-26T21:09:20.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.793+0000 7f108be00640 1 -- 172.21.15.26:0/3676982990 learned_addr learned my addr 172.21.15.26:0/3676982990 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:20.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.793+0000 7f1091400640 1 --2- 172.21.15.26:0/3676982990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f108c0d2aa0 0x7f108c15c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:20.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.793+0000 7f1090a00640 1 --2- 172.21.15.26:0/3676982990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f108c070c30 0x7f108c1f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:20.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.793+0000 7f108be00640 1 -- 172.21.15.26:0/3676982990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f108c0d2aa0 msgr2=0x7f108c15c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:20.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.793+0000 7f108be00640 1 --2- 172.21.15.26:0/3676982990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f108c0d2aa0 0x7f108c15c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:20.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.793+0000 7f108be00640 1 -- 172.21.15.26:0/3676982990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f108c070c30 msgr2=0x7f108c1f4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:20.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.793+0000 7f108be00640 1 --2- 172.21.15.26:0/3676982990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f108c070c30 0x7f108c1f4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:20.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.793+0000 7f108be00640 1 -- 172.21.15.26:0/3676982990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108c15efd0 con 0x7f108c0d2220 2024-04-26T21:09:20.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.793+0000 7f108be00640 1 --2- 172.21.15.26:0/3676982990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f108c0d2220 0x7f108c15e760 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f107800c400 tx=0x7f107800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:20.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.794+0000 7f1088c00640 1 -- 172.21.15.26:0/3676982990 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10780052d0 con 0x7f108c0d2220 2024-04-26T21:09:20.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.794+0000 7f1088c00640 1 -- 172.21.15.26:0/3676982990 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1078005490 con 0x7f108c0d2220 2024-04-26T21:09:20.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.794+0000 7f1088c00640 1 -- 172.21.15.26:0/3676982990 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078007260 con 0x7f108c0d2220 2024-04-26T21:09:20.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.794+0000 7f1091e00640 1 -- 172.21.15.26:0/3676982990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f108c15f2b0 con 0x7f108c0d2220 2024-04-26T21:09:20.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.794+0000 7f1091e00640 1 -- 172.21.15.26:0/3676982990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f108c159e10 con 0x7f108c0d2220 2024-04-26T21:09:20.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.795+0000 7f1091e00640 1 -- 172.21.15.26:0/3676982990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f108c0cb1f0 con 0x7f108c0d2220 2024-04-26T21:09:20.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.796+0000 7f1088c00640 1 -- 172.21.15.26:0/3676982990 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1078007400 con 0x7f108c0d2220 2024-04-26T21:09:20.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.797+0000 7f1088c00640 1 --2- 172.21.15.26:0/3676982990 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f106c06c8f0 0x7f106c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:20.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.797+0000 7f1088c00640 1 -- 172.21.15.26:0/3676982990 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f107808b050 con 0x7f108c0d2220 2024-04-26T21:09:20.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.798+0000 7f1090a00640 1 --2- 172.21.15.26:0/3676982990 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f106c06c8f0 0x7f106c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:20.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.798+0000 7f1090a00640 1 --2- 172.21.15.26:0/3676982990 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f106c06c8f0 0x7f106c06edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1080034000 tx=0x7f108002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:20.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:20.802+0000 7f1088c00640 1 -- 172.21.15.26:0/3676982990 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f108c0cb1f0 con 0x7f108c0d2220 2024-04-26T21:09:21.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.004+0000 7f1091e00640 1 -- 172.21.15.26:0/3676982990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f108c06a700 con 0x7f108c0d2220 2024-04-26T21:09:21.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.011+0000 7f1088c00640 1 -- 172.21.15.26:0/3676982990 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f1078059510 con 0x7f108c0d2220 2024-04-26T21:09:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.014+0000 7f1091e00640 1 -- 172.21.15.26:0/3676982990 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f106c06c8f0 msgr2=0x7f106c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.014+0000 7f1091e00640 1 --2- 172.21.15.26:0/3676982990 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f106c06c8f0 0x7f106c06edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1080034000 tx=0x7f108002e000 comp rx=0 tx=0).stop 2024-04-26T21:09:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.014+0000 7f1091e00640 1 -- 172.21.15.26:0/3676982990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f108c0d2220 msgr2=0x7f108c15e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.014+0000 7f1091e00640 1 --2- 172.21.15.26:0/3676982990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f108c0d2220 0x7f108c15e760 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f107800c400 tx=0x7f107800c8c0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.014+0000 7f1091e00640 1 -- 172.21.15.26:0/3676982990 shutdown_connections 2024-04-26T21:09:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.014+0000 7f1091e00640 1 --2- 172.21.15.26:0/3676982990 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f106c06c8f0 0x7f106c06edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.014+0000 7f1091e00640 1 --2- 172.21.15.26:0/3676982990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f108c0d2aa0 0x7f108c15c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.014+0000 7f1091e00640 1 --2- 172.21.15.26:0/3676982990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f108c0d2220 0x7f108c15e760 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.014+0000 7f1091e00640 1 --2- 172.21.15.26:0/3676982990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f108c070c30 0x7f108c1f4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.014+0000 7f1091e00640 1 -- 172.21.15.26:0/3676982990 >> 172.21.15.26:0/3676982990 conn(0x7f108c0d2ed0 msgr2=0x7f108c0caae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:21.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.014+0000 7f1091e00640 1 -- 172.21.15.26:0/3676982990 shutdown_connections 2024-04-26T21:09:21.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.015+0000 7f1091e00640 1 -- 172.21.15.26:0/3676982990 wait complete. 2024-04-26T21:09:21.035 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:09:21.035 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:09:21.154 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:09:21.267 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:09:21.267 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:09:21.267 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:09:21.329 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:09:21.329 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:09:21.385 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:09:21.385 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:09:21.438 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:09:21.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.650+0000 7f08cde00640 1 Processor -- start 2024-04-26T21:09:21.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.650+0000 7f08cde00640 1 -- start start 2024-04-26T21:09:21.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.652+0000 7f08cde00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08c8073470 0x7f08c8073850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:21.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.652+0000 7f08cde00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f08c80b4d40 0x7f08c80d11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:21.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.652+0000 7f08cde00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08c806f7d0 con 0x7f08c806f340 2024-04-26T21:09:21.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.652+0000 7f08cde00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f08c806fab0 con 0x7f08c80b4d40 2024-04-26T21:09:21.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.652+0000 7f08cde00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f08c806fd90 con 0x7f08c8073470 2024-04-26T21:09:21.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.652+0000 7f08cca00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08c8073470 0x7f08c8073850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:21.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.652+0000 7f08cca00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08c8073470 0x7f08c8073850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.26:33364/0 (socket says 172.21.15.26:33364) 2024-04-26T21:09:21.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.652+0000 7f08cca00640 1 -- 172.21.15.26:0/745058972 learned_addr learned my addr 172.21.15.26:0/745058972 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:21.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.652+0000 7f08cd400640 1 --2- 172.21.15.26:0/745058972 >> v2:172.21.15.86:3300/0 conn(0x7f08c80b4d40 0x7f08c80d11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.653+0000 7f08cca00640 1 -- 172.21.15.26:0/745058972 >> v2:172.21.15.86:3300/0 conn(0x7f08c80b4d40 msgr2=0x7f08c80d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.653+0000 7f08cca00640 1 --2- 172.21.15.26:0/745058972 >> v2:172.21.15.86:3300/0 conn(0x7f08c80b4d40 0x7f08c80d11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.653+0000 7f08cca00640 1 -- 172.21.15.26:0/745058972 >> v1:172.21.15.86:6789/0 conn(0x7f08c806f340 legacy=0x7f08c815faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.653+0000 7f08cca00640 1 -- 172.21.15.26:0/745058972 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c8070070 con 0x7f08c8073470 2024-04-26T21:09:21.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.654+0000 7f08cca00640 1 --2- 172.21.15.26:0/745058972 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08c8073470 0x7f08c8073850 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009870 tx=0x7f08bc02c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3937d79e9d7b6ef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:21.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.655+0000 7f08c7400640 1 -- 172.21.15.26:0/745058972 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc038070 con 0x7f08c8073470 2024-04-26T21:09:21.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.655+0000 7f08c7400640 1 -- 172.21.15.26:0/745058972 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08bc031710 con 0x7f08c8073470 2024-04-26T21:09:21.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.655+0000 7f08c7400640 1 -- 172.21.15.26:0/745058972 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc031ae0 con 0x7f08c8073470 2024-04-26T21:09:21.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.655+0000 7f08cde00640 1 -- 172.21.15.26:0/745058972 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08c8073470 msgr2=0x7f08c8073850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:21.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.655+0000 7f08cde00640 1 --2- 172.21.15.26:0/745058972 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08c8073470 0x7f08c8073850 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009870 tx=0x7f08bc02c480 comp rx=0 tx=0).stop 2024-04-26T21:09:21.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.656+0000 7f08cde00640 1 -- 172.21.15.26:0/745058972 shutdown_connections 2024-04-26T21:09:21.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.656+0000 7f08cde00640 1 --2- 172.21.15.26:0/745058972 >> v2:172.21.15.86:3300/0 conn(0x7f08c80b4d40 0x7f08c80d11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.656+0000 7f08cde00640 1 --2- 172.21.15.26:0/745058972 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08c8073470 0x7f08c8073850 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.656+0000 7f08cde00640 1 -- 172.21.15.26:0/745058972 >> 172.21.15.26:0/745058972 conn(0x7f08c806c690 msgr2=0x7f08c806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:21.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.656+0000 7f08cde00640 1 -- 172.21.15.26:0/745058972 shutdown_connections 2024-04-26T21:09:21.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.656+0000 7f08cde00640 1 -- 172.21.15.26:0/745058972 wait complete. 2024-04-26T21:09:21.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.657+0000 7f08cde00640 1 Processor -- start 2024-04-26T21:09:21.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.658+0000 7f08cde00640 1 -- start start 2024-04-26T21:09:21.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.658+0000 7f08cde00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08c8073470 0x7f08c815e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:21.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.658+0000 7f08cde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08c80b4d40 0x7f08c8152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:21.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.659+0000 7f08cde00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08c8156b70 0x7f08c8152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:21.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.659+0000 7f08cde00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f08c8073d90 con 0x7f08c8073470 2024-04-26T21:09:21.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.659+0000 7f08cde00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08c815c4b0 con 0x7f08c80b4d40 2024-04-26T21:09:21.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.659+0000 7f08cde00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f08c8073f10 con 0x7f08c8156b70 2024-04-26T21:09:21.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.659+0000 7f08cca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08c8073470 0x7f08c815e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:21.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.659+0000 7f08cca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08c8073470 0x7f08c815e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50970/0 (socket says 172.21.15.26:50970) 2024-04-26T21:09:21.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.659+0000 7f08cca00640 1 -- 172.21.15.26:0/1700618814 learned_addr learned my addr 172.21.15.26:0/1700618814 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:21.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.659+0000 7f08cd400640 1 --2- 172.21.15.26:0/1700618814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08c8156b70 0x7f08c8152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:21.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.659+0000 7f08c7e00640 1 --2- 172.21.15.26:0/1700618814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08c80b4d40 0x7f08c8152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:21.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.660+0000 7f08cca00640 1 -- 172.21.15.26:0/1700618814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08c8156b70 msgr2=0x7f08c8152d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:21.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.660+0000 7f08cca00640 1 --2- 172.21.15.26:0/1700618814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08c8156b70 0x7f08c8152d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.660+0000 7f08cca00640 1 -- 172.21.15.26:0/1700618814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08c80b4d40 msgr2=0x7f08c8152820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:21.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.660+0000 7f08cca00640 1 --2- 172.21.15.26:0/1700618814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08c80b4d40 0x7f08c8152820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.660+0000 7f08cca00640 1 -- 172.21.15.26:0/1700618814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c815ebc0 con 0x7f08c8073470 2024-04-26T21:09:21.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.660+0000 7f08cca00640 1 --2- 172.21.15.26:0/1700618814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08c8073470 0x7f08c815e390 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f08bc000c00 tx=0x7f08bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:21.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.660+0000 7f08c4c00640 1 -- 172.21.15.26:0/1700618814 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc038040 con 0x7f08c8073470 2024-04-26T21:09:21.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.660+0000 7f08c4c00640 1 -- 172.21.15.26:0/1700618814 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08bc033830 con 0x7f08c8073470 2024-04-26T21:09:21.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.660+0000 7f08cde00640 1 -- 172.21.15.26:0/1700618814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c80b2900 con 0x7f08c8073470 2024-04-26T21:09:21.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.660+0000 7f08c4c00640 1 -- 172.21.15.26:0/1700618814 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc033c40 con 0x7f08c8073470 2024-04-26T21:09:21.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.661+0000 7f08cde00640 1 -- 172.21.15.26:0/1700618814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c80b2ec0 con 0x7f08c8073470 2024-04-26T21:09:21.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.662+0000 7f08cde00640 1 -- 172.21.15.26:0/1700618814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08c806a700 con 0x7f08c8073470 2024-04-26T21:09:21.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.663+0000 7f08c4c00640 1 -- 172.21.15.26:0/1700618814 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f08bc03e430 con 0x7f08c8073470 2024-04-26T21:09:21.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.664+0000 7f08c4c00640 1 --2- 172.21.15.26:0/1700618814 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f08a806c920 0x7f08a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:21.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.664+0000 7f08c4c00640 1 -- 172.21.15.26:0/1700618814 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f08bc0ac560 con 0x7f08c8073470 2024-04-26T21:09:21.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.664+0000 7f08c7e00640 1 --2- 172.21.15.26:0/1700618814 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f08a806c920 0x7f08a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:21.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.665+0000 7f08c7e00640 1 --2- 172.21.15.26:0/1700618814 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f08a806c920 0x7f08a806ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f08b4007fd0 tx=0x7f08b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:21.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.668+0000 7f08c4c00640 1 -- 172.21.15.26:0/1700618814 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f08bc07a8f0 con 0x7f08c8073470 2024-04-26T21:09:21.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.879+0000 7f08cde00640 1 -- 172.21.15.26:0/1700618814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08c8155690 con 0x7f08c8073470 2024-04-26T21:09:21.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.882+0000 7f08c4c00640 1 -- 172.21.15.26:0/1700618814 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16557 (secure 0 0 0) 0x7f08bc07a1f0 con 0x7f08c8073470 2024-04-26T21:09:21.884 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:21.884 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:08:39.177854+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:08:35.481169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:08:36.609498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:09:21.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.885+0000 7f08cde00640 1 -- 172.21.15.26:0/1700618814 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f08a806c920 msgr2=0x7f08a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:21.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.885+0000 7f08cde00640 1 --2- 172.21.15.26:0/1700618814 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f08a806c920 0x7f08a806ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f08b4007fd0 tx=0x7f08b4009040 comp rx=0 tx=0).stop 2024-04-26T21:09:21.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.885+0000 7f08cde00640 1 -- 172.21.15.26:0/1700618814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08c8073470 msgr2=0x7f08c815e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:21.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.885+0000 7f08cde00640 1 --2- 172.21.15.26:0/1700618814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08c8073470 0x7f08c815e390 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f08bc000c00 tx=0x7f08bc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.885+0000 7f08cde00640 1 -- 172.21.15.26:0/1700618814 shutdown_connections 2024-04-26T21:09:21.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.885+0000 7f08cde00640 1 --2- 172.21.15.26:0/1700618814 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f08a806c920 0x7f08a806ede0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.885+0000 7f08cde00640 1 --2- 172.21.15.26:0/1700618814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f08c8156b70 0x7f08c8152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.885+0000 7f08cde00640 1 --2- 172.21.15.26:0/1700618814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08c80b4d40 0x7f08c8152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.885+0000 7f08cde00640 1 --2- 172.21.15.26:0/1700618814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08c8073470 0x7f08c815e390 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:21.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.885+0000 7f08cde00640 1 -- 172.21.15.26:0/1700618814 >> 172.21.15.26:0/1700618814 conn(0x7f08c806c690 msgr2=0x7f08c8154530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:21.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.885+0000 7f08cde00640 1 -- 172.21.15.26:0/1700618814 shutdown_connections 2024-04-26T21:09:21.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:21.885+0000 7f08cde00640 1 -- 172.21.15.26:0/1700618814 wait complete. 2024-04-26T21:09:21.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:08:31.322511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:08:35.481169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:08:36.609498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:09:21.907 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:09:22.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.120+0000 7f6b96a00640 1 Processor -- start 2024-04-26T21:09:22.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.121+0000 7f6b96a00640 1 -- start start 2024-04-26T21:09:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.122+0000 7f6b96a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6b9006f8c0 0x7f6b900b6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.122+0000 7f6b96a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b900b6650 0x7f6b90166c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.122+0000 7f6b96a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b90146c00 con 0x7f6b900b5d30 2024-04-26T21:09:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.122+0000 7f6b96a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f6b90073700 con 0x7f6b900b6650 2024-04-26T21:09:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.122+0000 7f6b96a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6b9006e420 con 0x7f6b9006f8c0 2024-04-26T21:09:22.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.123+0000 7f6b94a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b900b6650 0x7f6b90166c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:22.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.123+0000 7f6b94a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b900b6650 0x7f6b90166c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50972/0 (socket says 172.21.15.26:50972) 2024-04-26T21:09:22.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.123+0000 7f6b94a00640 1 -- 172.21.15.26:0/4173290636 learned_addr learned my addr 172.21.15.26:0/4173290636 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:22.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.123+0000 7f6b8f400640 1 --2- 172.21.15.26:0/4173290636 >> v2:172.21.15.86:3300/0 conn(0x7f6b9006f8c0 0x7f6b900b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:22.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.123+0000 7f6b94a00640 1 -- 172.21.15.26:0/4173290636 >> v2:172.21.15.86:3300/0 conn(0x7f6b9006f8c0 msgr2=0x7f6b900b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:22.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.123+0000 7f6b94a00640 1 --2- 172.21.15.26:0/4173290636 >> v2:172.21.15.86:3300/0 conn(0x7f6b9006f8c0 0x7f6b900b6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:22.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.123+0000 7f6b94a00640 1 -- 172.21.15.26:0/4173290636 >> v1:172.21.15.86:6789/0 conn(0x7f6b900b5d30 legacy=0x7f6b900b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:22.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.123+0000 7f6b94a00640 1 -- 172.21.15.26:0/4173290636 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b90072e60 con 0x7f6b900b6650 2024-04-26T21:09:22.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.124+0000 7f6b94a00640 1 --2- 172.21.15.26:0/4173290636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b900b6650 0x7f6b90166c90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6b90074520 tx=0x7f6b8802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d1fdadab862a18 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:22.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.125+0000 7f6b8ea00640 1 -- 172.21.15.26:0/4173290636 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88037070 con 0x7f6b900b6650 2024-04-26T21:09:22.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.125+0000 7f6b8ea00640 1 -- 172.21.15.26:0/4173290636 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b88032440 con 0x7f6b900b6650 2024-04-26T21:09:22.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.125+0000 7f6b8ea00640 1 -- 172.21.15.26:0/4173290636 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88032810 con 0x7f6b900b6650 2024-04-26T21:09:22.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.125+0000 7f6b96a00640 1 -- 172.21.15.26:0/4173290636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b900b6650 msgr2=0x7f6b90166c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:22.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.125+0000 7f6b96a00640 1 --2- 172.21.15.26:0/4173290636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b900b6650 0x7f6b90166c90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6b90074520 tx=0x7f6b8802a230 comp rx=0 tx=0).stop 2024-04-26T21:09:22.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.126+0000 7f6b96a00640 1 -- 172.21.15.26:0/4173290636 shutdown_connections 2024-04-26T21:09:22.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.126+0000 7f6b96a00640 1 --2- 172.21.15.26:0/4173290636 >> v2:172.21.15.86:3300/0 conn(0x7f6b9006f8c0 0x7f6b900b6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:22.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.126+0000 7f6b96a00640 1 --2- 172.21.15.26:0/4173290636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b900b6650 0x7f6b90166c90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:22.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.126+0000 7f6b96a00640 1 -- 172.21.15.26:0/4173290636 >> 172.21.15.26:0/4173290636 conn(0x7f6b900b2a80 msgr2=0x7f6b900b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:22.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.126+0000 7f6b96a00640 1 -- 172.21.15.26:0/4173290636 shutdown_connections 2024-04-26T21:09:22.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.126+0000 7f6b96a00640 1 -- 172.21.15.26:0/4173290636 wait complete. 2024-04-26T21:09:22.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.128+0000 7f6b96a00640 1 Processor -- start 2024-04-26T21:09:22.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.128+0000 7f6b96a00640 1 -- start start 2024-04-26T21:09:22.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.129+0000 7f6b96a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b9006f8c0 0x7f6b90142590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:22.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.129+0000 7f6b96a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b900b5d30 0x7f6b901f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:22.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.129+0000 7f6b96a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b900b6650 0x7f6b901f86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:22.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.129+0000 7f6b96a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6b900b4490 con 0x7f6b900b6650 2024-04-26T21:09:22.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.129+0000 7f6b96a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6b90073420 con 0x7f6b900b5d30 2024-04-26T21:09:22.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.129+0000 7f6b96a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6b90072a80 con 0x7f6b9006f8c0 2024-04-26T21:09:22.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.129+0000 7f6b94a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b900b6650 0x7f6b901f86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:22.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.129+0000 7f6b94a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b900b6650 0x7f6b901f86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50988/0 (socket says 172.21.15.26:50988) 2024-04-26T21:09:22.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.129+0000 7f6b94a00640 1 -- 172.21.15.26:0/1204805472 learned_addr learned my addr 172.21.15.26:0/1204805472 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:22.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.129+0000 7f6b8f400640 1 --2- 172.21.15.26:0/1204805472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b900b5d30 0x7f6b901f81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:22.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.129+0000 7f6b8fe00640 1 --2- 172.21.15.26:0/1204805472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b9006f8c0 0x7f6b90142590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:22.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.130+0000 7f6b94a00640 1 -- 172.21.15.26:0/1204805472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b9006f8c0 msgr2=0x7f6b90142590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:22.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.130+0000 7f6b94a00640 1 --2- 172.21.15.26:0/1204805472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b9006f8c0 0x7f6b90142590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:22.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.130+0000 7f6b94a00640 1 -- 172.21.15.26:0/1204805472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b900b5d30 msgr2=0x7f6b901f81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:22.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.130+0000 7f6b94a00640 1 --2- 172.21.15.26:0/1204805472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b900b5d30 0x7f6b901f81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:22.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.130+0000 7f6b94a00640 1 -- 172.21.15.26:0/1204805472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b900cff70 con 0x7f6b900b6650 2024-04-26T21:09:22.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.130+0000 7f6b8fe00640 1 --2- 172.21.15.26:0/1204805472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b9006f8c0 0x7f6b90142590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:22.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.130+0000 7f6b94a00640 1 --2- 172.21.15.26:0/1204805472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b900b6650 0x7f6b901f86f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6b880090b0 tx=0x7f6b880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:22.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.131+0000 7f6b87e00640 1 -- 172.21.15.26:0/1204805472 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88041020 con 0x7f6b900b6650 2024-04-26T21:09:22.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.131+0000 7f6b87e00640 1 -- 172.21.15.26:0/1204805472 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b880306c0 con 0x7f6b900b6650 2024-04-26T21:09:22.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.131+0000 7f6b87e00640 1 -- 172.21.15.26:0/1204805472 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88037040 con 0x7f6b900b6650 2024-04-26T21:09:22.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.131+0000 7f6b96a00640 1 -- 172.21.15.26:0/1204805472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b901443c0 con 0x7f6b900b6650 2024-04-26T21:09:22.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.131+0000 7f6b96a00640 1 -- 172.21.15.26:0/1204805472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b90144980 con 0x7f6b900b6650 2024-04-26T21:09:22.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.132+0000 7f6b96a00640 1 -- 172.21.15.26:0/1204805472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b9006a700 con 0x7f6b900b6650 2024-04-26T21:09:22.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.133+0000 7f6b87e00640 1 -- 172.21.15.26:0/1204805472 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b88044020 con 0x7f6b900b6650 2024-04-26T21:09:22.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.134+0000 7f6b87e00640 1 --2- 172.21.15.26:0/1204805472 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6b7006c8f0 0x7f6b7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:22.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.134+0000 7f6b87e00640 1 -- 172.21.15.26:0/1204805472 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6b880ac360 con 0x7f6b900b6650 2024-04-26T21:09:22.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.134+0000 7f6b8fe00640 1 --2- 172.21.15.26:0/1204805472 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6b7006c8f0 0x7f6b7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:22.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.135+0000 7f6b8fe00640 1 --2- 172.21.15.26:0/1204805472 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6b7006c8f0 0x7f6b7006edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6b900b32b0 tx=0x7f6b80009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:22.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.138+0000 7f6b87e00640 1 -- 172.21.15.26:0/1204805472 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6b8804b030 con 0x7f6b900b6650 2024-04-26T21:09:22.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.341+0000 7f6b96a00640 1 -- 172.21.15.26:0/1204805472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6b90144130 con 0x7f6b900b6650 2024-04-26T21:09:22.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.342+0000 7f6b87e00640 1 -- 172.21.15.26:0/1204805472 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f6b8807a780 con 0x7f6b900b6650 2024-04-26T21:09:22.344 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-26T21:09:22.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.345+0000 7f6b96a00640 1 -- 172.21.15.26:0/1204805472 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6b7006c8f0 msgr2=0x7f6b7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:22.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.345+0000 7f6b96a00640 1 --2- 172.21.15.26:0/1204805472 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6b7006c8f0 0x7f6b7006edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6b900b32b0 tx=0x7f6b80009040 comp rx=0 tx=0).stop 2024-04-26T21:09:22.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.345+0000 7f6b96a00640 1 -- 172.21.15.26:0/1204805472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b900b6650 msgr2=0x7f6b901f86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:22.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.345+0000 7f6b96a00640 1 --2- 172.21.15.26:0/1204805472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b900b6650 0x7f6b901f86f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6b880090b0 tx=0x7f6b880046c0 comp rx=0 tx=0).stop 2024-04-26T21:09:22.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.345+0000 7f6b96a00640 1 -- 172.21.15.26:0/1204805472 shutdown_connections 2024-04-26T21:09:22.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.345+0000 7f6b96a00640 1 --2- 172.21.15.26:0/1204805472 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6b7006c8f0 0x7f6b7006edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:22.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.345+0000 7f6b96a00640 1 --2- 172.21.15.26:0/1204805472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b900b6650 0x7f6b901f86f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:22.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.345+0000 7f6b96a00640 1 --2- 172.21.15.26:0/1204805472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b900b5d30 0x7f6b901f81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:22.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.345+0000 7f6b96a00640 1 --2- 172.21.15.26:0/1204805472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b9006f8c0 0x7f6b90142590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:22.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.345+0000 7f6b96a00640 1 -- 172.21.15.26:0/1204805472 >> 172.21.15.26:0/1204805472 conn(0x7f6b900b2a80 msgr2=0x7f6b900b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:22.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.345+0000 7f6b96a00640 1 -- 172.21.15.26:0/1204805472 shutdown_connections 2024-04-26T21:09:22.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.345+0000 7f6b96a00640 1 -- 172.21.15.26:0/1204805472 wait complete. 2024-04-26T21:09:22.366 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:09:22.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.583+0000 7f959f200640 1 Processor -- start 2024-04-26T21:09:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.583+0000 7f959f200640 1 -- start start 2024-04-26T21:09:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.584+0000 7f959f200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95a015f290 0x7f95a00d5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.584+0000 7f959f200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95a00d3870 con 0x7f95a00d5c40 2024-04-26T21:09:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.584+0000 7f959f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95a00d3b50 con 0x7f95a0163570 2024-04-26T21:09:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.584+0000 7f959f200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f95a00d3e30 con 0x7f95a015f290 2024-04-26T21:09:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.585+0000 7f959d400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f95a00d5c40 0x7f95a0162e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54566/0 (socket says 172.21.15.26:54566) 2024-04-26T21:09:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.585+0000 7f959d400640 1 -- 172.21.15.26:0/3911132047 learned_addr learned my addr 172.21.15.26:0/3911132047 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.585+0000 7f959de00640 1 --2- 172.21.15.26:0/3911132047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95a015f290 0x7f95a00d5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.585+0000 7f959ca00640 1 -- 172.21.15.26:0/3911132047 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2881138575 0 0) 0x7f95a00d3870 con 0x7f95a00d5c40 2024-04-26T21:09:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.585+0000 7f959ca00640 1 -- 172.21.15.26:0/3911132047 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9584003520 con 0x7f95a00d5c40 2024-04-26T21:09:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.586+0000 7f959ca00640 1 -- 172.21.15.26:0/3911132047 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2532005863 0 0) 0x7f9584003520 con 0x7f95a00d5c40 2024-04-26T21:09:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.586+0000 7f959ca00640 1 -- 172.21.15.26:0/3911132047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95a015f290 msgr2=0x7f95a00d5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.586+0000 7f959ca00640 1 --2- 172.21.15.26:0/3911132047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95a015f290 0x7f95a00d5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.586+0000 7f959ca00640 1 -- 172.21.15.26:0/3911132047 >> v1:172.21.15.86:6789/0 conn(0x7f95a0163570 legacy=0x7f95a00b8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.586+0000 7f959ca00640 1 -- 172.21.15.26:0/3911132047 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a00b7e40 con 0x7f95a00d5c40 2024-04-26T21:09:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.586+0000 7f959ca00640 1 -- 172.21.15.26:0/3911132047 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f958c001670 con 0x7f95a00d5c40 2024-04-26T21:09:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.586+0000 7f959ca00640 1 -- 172.21.15.26:0/3911132047 >> v1:172.21.15.26:6789/0 conn(0x7f95a00d5c40 legacy=0x7f95a0162e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.586+0000 7f959ca00640 1 --2- 172.21.15.26:0/3911132047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95840039e0 0x7f9584003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.586+0000 7f959ca00640 1 --2- 172.21.15.26:0/3911132047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9584004390 0x7f9584004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.586+0000 7f959ca00640 1 --2- 172.21.15.26:0/3911132047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95a015f290 0x7f95840050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.586+0000 7f959ca00640 1 -- 172.21.15.26:0/3911132047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f95a00d3e30 con 0x7f95840039e0 2024-04-26T21:09:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.586+0000 7f959ca00640 1 -- 172.21.15.26:0/3911132047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f958400b9c0 con 0x7f9584004390 2024-04-26T21:09:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.586+0000 7f959ca00640 1 -- 172.21.15.26:0/3911132047 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f958400bbc0 con 0x7f95a015f290 2024-04-26T21:09:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.586+0000 7f959de00640 1 --2- 172.21.15.26:0/3911132047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95840039e0 0x7f9584003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.587+0000 7f959d400640 1 --2- 172.21.15.26:0/3911132047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9584004390 0x7f9584004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.587+0000 7f959de00640 1 -- 172.21.15.26:0/3911132047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95a015f290 msgr2=0x7f95840050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.587+0000 7f959de00640 1 --2- 172.21.15.26:0/3911132047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95a015f290 0x7f95840050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.587+0000 7f959de00640 1 -- 172.21.15.26:0/3911132047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9584004390 msgr2=0x7f9584004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.587+0000 7f959de00640 1 --2- 172.21.15.26:0/3911132047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9584004390 0x7f9584004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.587+0000 7f959de00640 1 -- 172.21.15.26:0/3911132047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9584003520 con 0x7f95840039e0 2024-04-26T21:09:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.588+0000 7f959de00640 1 --2- 172.21.15.26:0/3911132047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95840039e0 0x7f9584003dc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9590009ac0 tx=0x7f9584005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7542bd7d0c437da8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.588+0000 7f959ca00640 1 -- 172.21.15.26:0/3911132047 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590036070 con 0x7f95840039e0 2024-04-26T21:09:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.588+0000 7f959ca00640 1 -- 172.21.15.26:0/3911132047 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9590031400 con 0x7f95840039e0 2024-04-26T21:09:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.588+0000 7f959ca00640 1 -- 172.21.15.26:0/3911132047 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590009690 con 0x7f95840039e0 2024-04-26T21:09:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.588+0000 7f959f200640 1 -- 172.21.15.26:0/3911132047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95840039e0 msgr2=0x7f9584003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.588+0000 7f959f200640 1 --2- 172.21.15.26:0/3911132047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95840039e0 0x7f9584003dc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9590009ac0 tx=0x7f9584005830 comp rx=0 tx=0).stop 2024-04-26T21:09:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.589+0000 7f959e800640 1 -- 172.21.15.26:0/3911132047 reap_dead start 2024-04-26T21:09:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.589+0000 7f959f200640 1 -- 172.21.15.26:0/3911132047 shutdown_connections 2024-04-26T21:09:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.589+0000 7f959f200640 1 -- 172.21.15.26:0/3911132047 >> 172.21.15.26:0/3911132047 conn(0x7f95a009e380 msgr2=0x7f95a00d4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.589+0000 7f959f200640 1 -- 172.21.15.26:0/3911132047 shutdown_connections 2024-04-26T21:09:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.589+0000 7f959f200640 1 -- 172.21.15.26:0/3911132047 wait complete. 2024-04-26T21:09:22.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.590+0000 7f959f200640 1 Processor -- start 2024-04-26T21:09:22.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.591+0000 7f959f200640 1 -- start start 2024-04-26T21:09:22.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.592+0000 7f959f200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95840039e0 0x7f95a0153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:22.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.592+0000 7f959f200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95a0156150 0x7f95a01565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:22.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.592+0000 7f959f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a01e9610 0x7f95a01e9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:22.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.592+0000 7f959f200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f95a015f670 con 0x7f95a0156150 2024-04-26T21:09:22.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.592+0000 7f959f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95a00d3540 con 0x7f95a01e9610 2024-04-26T21:09:22.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.592+0000 7f959f200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f95a00d5700 con 0x7f95840039e0 2024-04-26T21:09:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.592+0000 7f959d400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95a0156150 0x7f95a01565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.592+0000 7f959d400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95a0156150 0x7f95a01565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51014/0 (socket says 172.21.15.26:51014) 2024-04-26T21:09:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.592+0000 7f959d400640 1 -- 172.21.15.26:0/2692813238 learned_addr learned my addr 172.21.15.26:0/2692813238 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.592+0000 7f959e800640 1 --2- 172.21.15.26:0/2692813238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a01e9610 0x7f95a01e9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.592+0000 7f959de00640 1 --2- 172.21.15.26:0/2692813238 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95840039e0 0x7f95a0153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.592+0000 7f959d400640 1 -- 172.21.15.26:0/2692813238 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95840039e0 msgr2=0x7f95a0153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.592+0000 7f959d400640 1 --2- 172.21.15.26:0/2692813238 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95840039e0 0x7f95a0153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.592+0000 7f959d400640 1 -- 172.21.15.26:0/2692813238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a01e9610 msgr2=0x7f95a01e9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.592+0000 7f959d400640 1 --2- 172.21.15.26:0/2692813238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a01e9610 0x7f95a01e9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.592+0000 7f959d400640 1 -- 172.21.15.26:0/2692813238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a0161fc0 con 0x7f95a0156150 2024-04-26T21:09:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.592+0000 7f959de00640 1 --2- 172.21.15.26:0/2692813238 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95840039e0 0x7f95a0153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.592+0000 7f959e800640 1 --2- 172.21.15.26:0/2692813238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a01e9610 0x7f95a01e9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.593+0000 7f959d400640 1 --2- 172.21.15.26:0/2692813238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95a0156150 0x7f95a01565d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f958c00e780 tx=0x7f958c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.593+0000 7f9596000640 1 -- 172.21.15.26:0/2692813238 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c00c870 con 0x7f95a0156150 2024-04-26T21:09:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.593+0000 7f9596000640 1 -- 172.21.15.26:0/2692813238 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f958c018070 con 0x7f95a0156150 2024-04-26T21:09:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.593+0000 7f9596000640 1 -- 172.21.15.26:0/2692813238 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c00cc30 con 0x7f95a0156150 2024-04-26T21:09:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.593+0000 7f959f200640 1 -- 172.21.15.26:0/2692813238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a01ea250 con 0x7f95a0156150 2024-04-26T21:09:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.593+0000 7f959f200640 1 -- 172.21.15.26:0/2692813238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a0153e50 con 0x7f95a0156150 2024-04-26T21:09:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.594+0000 7f959f200640 1 -- 172.21.15.26:0/2692813238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95a006a700 con 0x7f95a0156150 2024-04-26T21:09:22.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.595+0000 7f9596000640 1 -- 172.21.15.26:0/2692813238 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f958c01d460 con 0x7f95a0156150 2024-04-26T21:09:22.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.596+0000 7f9596000640 1 --2- 172.21.15.26:0/2692813238 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f958006c8f0 0x7f958006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:22.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.596+0000 7f9596000640 1 -- 172.21.15.26:0/2692813238 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f958c08ba70 con 0x7f95a0156150 2024-04-26T21:09:22.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.597+0000 7f959de00640 1 --2- 172.21.15.26:0/2692813238 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f958006c8f0 0x7f958006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:22.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.597+0000 7f959de00640 1 --2- 172.21.15.26:0/2692813238 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f958006c8f0 0x7f958006edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9590009ac0 tx=0x7f9590008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:22.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.601+0000 7f9596000640 1 -- 172.21.15.26:0/2692813238 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f958c059e90 con 0x7f95a0156150 2024-04-26T21:09:22.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.812+0000 7f959f200640 1 -- 172.21.15.26:0/2692813238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f95a00b76e0 con 0x7f95a0156150 2024-04-26T21:09:22.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.813+0000 7f9596000640 1 -- 172.21.15.26:0/2692813238 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f958c059790 con 0x7f95a0156150 2024-04-26T21:09:22.814 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-26T21:09:22.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.815+0000 7f959f200640 1 -- 172.21.15.26:0/2692813238 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f958006c8f0 msgr2=0x7f958006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:22.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.815+0000 7f959f200640 1 --2- 172.21.15.26:0/2692813238 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f958006c8f0 0x7f958006edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9590009ac0 tx=0x7f9590008030 comp rx=0 tx=0).stop 2024-04-26T21:09:22.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.815+0000 7f959f200640 1 -- 172.21.15.26:0/2692813238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95a0156150 msgr2=0x7f95a01565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:22.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.815+0000 7f959f200640 1 --2- 172.21.15.26:0/2692813238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95a0156150 0x7f95a01565d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f958c00e780 tx=0x7f958c00ec40 comp rx=0 tx=0).stop 2024-04-26T21:09:22.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.816+0000 7f959f200640 1 -- 172.21.15.26:0/2692813238 shutdown_connections 2024-04-26T21:09:22.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.816+0000 7f959f200640 1 --2- 172.21.15.26:0/2692813238 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f958006c8f0 0x7f958006edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:22.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.816+0000 7f959f200640 1 --2- 172.21.15.26:0/2692813238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a01e9610 0x7f95a01e9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:22.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.816+0000 7f959f200640 1 --2- 172.21.15.26:0/2692813238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95a0156150 0x7f95a01565d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:22.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.816+0000 7f959f200640 1 --2- 172.21.15.26:0/2692813238 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95840039e0 0x7f95a0153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:22.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.816+0000 7f959f200640 1 -- 172.21.15.26:0/2692813238 >> 172.21.15.26:0/2692813238 conn(0x7f95a009e380 msgr2=0x7f95a00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:22.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.816+0000 7f959f200640 1 -- 172.21.15.26:0/2692813238 shutdown_connections 2024-04-26T21:09:22.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:22.816+0000 7f959f200640 1 -- 172.21.15.26:0/2692813238 wait complete. 2024-04-26T21:09:22.837 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:09:23.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.043+0000 7ff8fd400640 1 Processor -- start 2024-04-26T21:09:23.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.043+0000 7ff8fd400640 1 -- start start 2024-04-26T21:09:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.044+0000 7ff8fd400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff8f814d120 0x7ff8f80cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.045+0000 7ff8fd400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff8f806f9f0 0x7ff8f80b4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.045+0000 7ff8fd400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f8073c50 con 0x7ff8f8073270 2024-04-26T21:09:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.045+0000 7ff8fd400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ff8f80cf5a0 con 0x7ff8f806f9f0 2024-04-26T21:09:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.045+0000 7ff8fd400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff8f80cec70 con 0x7ff8f814d120 2024-04-26T21:09:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.045+0000 7ff8fca00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff8f806f9f0 0x7ff8f80b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.045+0000 7ff8fca00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff8f806f9f0 0x7ff8f80b4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51022/0 (socket says 172.21.15.26:51022) 2024-04-26T21:09:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.045+0000 7ff8fca00640 1 -- 172.21.15.26:0/2168594035 learned_addr learned my addr 172.21.15.26:0/2168594035 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.045+0000 7ff8f7e00640 1 --2- 172.21.15.26:0/2168594035 >> v2:172.21.15.86:3300/0 conn(0x7ff8f814d120 0x7ff8f80cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.046+0000 7ff8fca00640 1 -- 172.21.15.26:0/2168594035 >> v2:172.21.15.86:3300/0 conn(0x7ff8f814d120 msgr2=0x7ff8f80cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.046+0000 7ff8fca00640 1 --2- 172.21.15.26:0/2168594035 >> v2:172.21.15.86:3300/0 conn(0x7ff8f814d120 0x7ff8f80cfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.046+0000 7ff8fca00640 1 -- 172.21.15.26:0/2168594035 >> v1:172.21.15.86:6789/0 conn(0x7ff8f8073270 legacy=0x7ff8f806f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.046+0000 7ff8fca00640 1 -- 172.21.15.26:0/2168594035 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f80cef50 con 0x7ff8f806f9f0 2024-04-26T21:09:23.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.048+0000 7ff8fca00640 1 --2- 172.21.15.26:0/2168594035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f806f9f0 0x7ff8f80b4900 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009e50 tx=0x7ff8f002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee067aca9eb8e453 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:23.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.048+0000 7ff8f6a00640 1 -- 172.21.15.26:0/2168594035 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0036070 con 0x7ff8f806f9f0 2024-04-26T21:09:23.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.048+0000 7ff8f6a00640 1 -- 172.21.15.26:0/2168594035 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f0031600 con 0x7ff8f806f9f0 2024-04-26T21:09:23.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.048+0000 7ff8f6a00640 1 -- 172.21.15.26:0/2168594035 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f00319d0 con 0x7ff8f806f9f0 2024-04-26T21:09:23.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.049+0000 7ff8fd400640 1 -- 172.21.15.26:0/2168594035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f806f9f0 msgr2=0x7ff8f80b4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:23.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.049+0000 7ff8fd400640 1 --2- 172.21.15.26:0/2168594035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f806f9f0 0x7ff8f80b4900 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009e50 tx=0x7ff8f002a6d0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.049+0000 7ff8fd400640 1 -- 172.21.15.26:0/2168594035 shutdown_connections 2024-04-26T21:09:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.049+0000 7ff8fd400640 1 --2- 172.21.15.26:0/2168594035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f806f9f0 0x7ff8f80b4900 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.049+0000 7ff8fd400640 1 --2- 172.21.15.26:0/2168594035 >> v2:172.21.15.86:3300/0 conn(0x7ff8f814d120 0x7ff8f80cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.049+0000 7ff8fd400640 1 -- 172.21.15.26:0/2168594035 >> 172.21.15.26:0/2168594035 conn(0x7ff8f80cd7d0 msgr2=0x7ff8f80cdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.049+0000 7ff8fd400640 1 -- 172.21.15.26:0/2168594035 shutdown_connections 2024-04-26T21:09:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.049+0000 7ff8fd400640 1 -- 172.21.15.26:0/2168594035 wait complete. 2024-04-26T21:09:23.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.051+0000 7ff8fd400640 1 Processor -- start 2024-04-26T21:09:23.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.051+0000 7ff8fd400640 1 -- start start 2024-04-26T21:09:23.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.052+0000 7ff8fd400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8f806f9f0 0x7ff8f8158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:23.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.052+0000 7ff8fd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f8073270 0x7ff8f8164b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:23.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.052+0000 7ff8fd400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f814d120 0x7ff8f81471b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:23.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.052+0000 7ff8fd400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f8072e10 con 0x7ff8f814d120 2024-04-26T21:09:23.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.052+0000 7ff8fd400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f8072c90 con 0x7ff8f8073270 2024-04-26T21:09:23.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.052+0000 7ff8fd400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f8073700 con 0x7ff8f806f9f0 2024-04-26T21:09:23.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.052+0000 7ff8fca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f814d120 0x7ff8f81471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:23.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.052+0000 7ff8fca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f814d120 0x7ff8f81471b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51032/0 (socket says 172.21.15.26:51032) 2024-04-26T21:09:23.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.052+0000 7ff8fca00640 1 -- 172.21.15.26:0/4231292141 learned_addr learned my addr 172.21.15.26:0/4231292141 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:23.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.053+0000 7ff8f7400640 1 --2- 172.21.15.26:0/4231292141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f8073270 0x7ff8f8164b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:23.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.053+0000 7ff8f7e00640 1 --2- 172.21.15.26:0/4231292141 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8f806f9f0 0x7ff8f8158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:23.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.053+0000 7ff8fca00640 1 -- 172.21.15.26:0/4231292141 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8f806f9f0 msgr2=0x7ff8f8158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:23.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.053+0000 7ff8fca00640 1 --2- 172.21.15.26:0/4231292141 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8f806f9f0 0x7ff8f8158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.053+0000 7ff8fca00640 1 -- 172.21.15.26:0/4231292141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f8073270 msgr2=0x7ff8f8164b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:23.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.053+0000 7ff8fca00640 1 --2- 172.21.15.26:0/4231292141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f8073270 0x7ff8f8164b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.053+0000 7ff8fca00640 1 -- 172.21.15.26:0/4231292141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f8159350 con 0x7ff8f814d120 2024-04-26T21:09:23.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.053+0000 7ff8fca00640 1 --2- 172.21.15.26:0/4231292141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f814d120 0x7ff8f81471b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009f70 tx=0x7ff8f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:23.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.054+0000 7ff8efe00640 1 -- 172.21.15.26:0/4231292141 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0041020 con 0x7ff8f814d120 2024-04-26T21:09:23.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.054+0000 7ff8efe00640 1 -- 172.21.15.26:0/4231292141 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f0008730 con 0x7ff8f814d120 2024-04-26T21:09:23.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.054+0000 7ff8fd400640 1 -- 172.21.15.26:0/4231292141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f8165460 con 0x7ff8f814d120 2024-04-26T21:09:23.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.054+0000 7ff8efe00640 1 -- 172.21.15.26:0/4231292141 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0036040 con 0x7ff8f814d120 2024-04-26T21:09:23.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.054+0000 7ff8fd400640 1 -- 172.21.15.26:0/4231292141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f8165a20 con 0x7ff8f814d120 2024-04-26T21:09:23.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.055+0000 7ff8fd400640 1 -- 172.21.15.26:0/4231292141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f80506c0 con 0x7ff8f814d120 2024-04-26T21:09:23.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.056+0000 7ff8efe00640 1 -- 172.21.15.26:0/4231292141 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8f0044020 con 0x7ff8f814d120 2024-04-26T21:09:23.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.058+0000 7ff8efe00640 1 --2- 172.21.15.26:0/4231292141 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff8d406c8f0 0x7ff8d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:23.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.058+0000 7ff8efe00640 1 -- 172.21.15.26:0/4231292141 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff8f00ac810 con 0x7ff8f814d120 2024-04-26T21:09:23.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.058+0000 7ff8f7e00640 1 --2- 172.21.15.26:0/4231292141 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff8d406c8f0 0x7ff8d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.059+0000 7ff8f7e00640 1 --2- 172.21.15.26:0/4231292141 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff8d406c8f0 0x7ff8d406edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009a20 tx=0x7ff8e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:23.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.063+0000 7ff8efe00640 1 -- 172.21.15.26:0/4231292141 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff8f004b070 con 0x7ff8f814d120 2024-04-26T21:09:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.294+0000 7ff8fd400640 1 -- 172.21.15.26:0/4231292141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff8f80ce8b0 con 0x7ff8f814d120 2024-04-26T21:09:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.295+0000 7ff8efe00640 1 -- 172.21.15.26:0/4231292141 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7ff8f007aba0 con 0x7ff8f814d120 2024-04-26T21:09:23.297 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-26T21:09:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.298+0000 7ff8fd400640 1 -- 172.21.15.26:0/4231292141 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff8d406c8f0 msgr2=0x7ff8d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.298+0000 7ff8fd400640 1 --2- 172.21.15.26:0/4231292141 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff8d406c8f0 0x7ff8d406edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009a20 tx=0x7ff8e8008040 comp rx=0 tx=0).stop 2024-04-26T21:09:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.298+0000 7ff8fd400640 1 -- 172.21.15.26:0/4231292141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f814d120 msgr2=0x7ff8f81471b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.298+0000 7ff8fd400640 1 --2- 172.21.15.26:0/4231292141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f814d120 0x7ff8f81471b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009f70 tx=0x7ff8f0004a40 comp rx=0 tx=0).stop 2024-04-26T21:09:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.298+0000 7ff8fd400640 1 -- 172.21.15.26:0/4231292141 shutdown_connections 2024-04-26T21:09:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.298+0000 7ff8fd400640 1 --2- 172.21.15.26:0/4231292141 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff8d406c8f0 0x7ff8d406edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.298+0000 7ff8fd400640 1 --2- 172.21.15.26:0/4231292141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8f814d120 0x7ff8f81471b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.298+0000 7ff8fd400640 1 --2- 172.21.15.26:0/4231292141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f8073270 0x7ff8f8164b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.298+0000 7ff8fd400640 1 --2- 172.21.15.26:0/4231292141 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8f806f9f0 0x7ff8f8158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.298+0000 7ff8fd400640 1 -- 172.21.15.26:0/4231292141 >> 172.21.15.26:0/4231292141 conn(0x7ff8f80cd7d0 msgr2=0x7ff8f80ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.298+0000 7ff8fd400640 1 -- 172.21.15.26:0/4231292141 shutdown_connections 2024-04-26T21:09:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.298+0000 7ff8fd400640 1 -- 172.21.15.26:0/4231292141 wait complete. 2024-04-26T21:09:23.320 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:09:23.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.518+0000 7f30bbe00640 1 Processor -- start 2024-04-26T21:09:23.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.518+0000 7f30bbe00640 1 -- start start 2024-04-26T21:09:23.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.519+0000 7f30bbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f30bc145c20 0x7f30bc0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:23.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.520+0000 7f30bbe00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30bc0cf690 con 0x7f30bc146450 2024-04-26T21:09:23.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.520+0000 7f30bbe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30bc0cf930 con 0x7f30bc0d1840 2024-04-26T21:09:23.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.520+0000 7f30bbe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f30bc0ced10 con 0x7f30bc145c20 2024-04-26T21:09:23.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.520+0000 7f30ba000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f30bc146450 0x7f30bc0d1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54582/0 (socket says 172.21.15.26:54582) 2024-04-26T21:09:23.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.520+0000 7f30ba000640 1 -- 172.21.15.26:0/3376236329 learned_addr learned my addr 172.21.15.26:0/3376236329 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:23.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.520+0000 7f30b9600640 1 -- 172.21.15.26:0/3376236329 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2416369622 0 0) 0x7f30bc0cf690 con 0x7f30bc146450 2024-04-26T21:09:23.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.520+0000 7f30baa00640 1 --2- 172.21.15.26:0/3376236329 >> v2:172.21.15.86:3300/0 conn(0x7f30bc145c20 0x7f30bc0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:23.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.520+0000 7f30b9600640 1 -- 172.21.15.26:0/3376236329 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30a0003520 con 0x7f30bc146450 2024-04-26T21:09:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.521+0000 7f30b9600640 1 -- 172.21.15.26:0/3376236329 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2309021583 0 0) 0x7f30a0003520 con 0x7f30bc146450 2024-04-26T21:09:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.521+0000 7f30b9600640 1 -- 172.21.15.26:0/3376236329 >> v2:172.21.15.86:3300/0 conn(0x7f30bc145c20 msgr2=0x7f30bc0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.521+0000 7f30b9600640 1 --2- 172.21.15.26:0/3376236329 >> v2:172.21.15.86:3300/0 conn(0x7f30bc145c20 0x7f30bc0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.521+0000 7f30b9600640 1 -- 172.21.15.26:0/3376236329 >> v1:172.21.15.86:6789/0 conn(0x7f30bc0d1840 legacy=0x7f30bc071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.521+0000 7f30b9600640 1 -- 172.21.15.26:0/3376236329 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30bc0cfc10 con 0x7f30bc146450 2024-04-26T21:09:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.521+0000 7f30b9600640 1 -- 172.21.15.26:0/3376236329 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f30a8001690 con 0x7f30bc146450 2024-04-26T21:09:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.521+0000 7f30b9600640 1 -- 172.21.15.26:0/3376236329 >> v1:172.21.15.26:6789/0 conn(0x7f30bc146450 legacy=0x7f30bc0d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.521+0000 7f30b9600640 1 --2- 172.21.15.26:0/3376236329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30a00039b0 0x7f30a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.521+0000 7f30b9600640 1 --2- 172.21.15.26:0/3376236329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a0004800 0x7f30a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.521+0000 7f30b9600640 1 --2- 172.21.15.26:0/3376236329 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f30a00051c0 0x7f30a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.521+0000 7f30b9600640 1 -- 172.21.15.26:0/3376236329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f30a000c0e0 con 0x7f30a0004800 2024-04-26T21:09:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.521+0000 7f30b9600640 1 -- 172.21.15.26:0/3376236329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f30a000c290 con 0x7f30a00039b0 2024-04-26T21:09:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.521+0000 7f30b9600640 1 -- 172.21.15.26:0/3376236329 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f30a000c490 con 0x7f30a00051c0 2024-04-26T21:09:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.521+0000 7f30ba000640 1 --2- 172.21.15.26:0/3376236329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a0004800 0x7f30a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:23.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.522+0000 7f30baa00640 1 --2- 172.21.15.26:0/3376236329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30a00039b0 0x7f30a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:23.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.522+0000 7f30ba000640 1 -- 172.21.15.26:0/3376236329 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f30a00051c0 msgr2=0x7f30a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:23.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.522+0000 7f30ba000640 1 --2- 172.21.15.26:0/3376236329 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f30a00051c0 0x7f30a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.522+0000 7f30ba000640 1 -- 172.21.15.26:0/3376236329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30a00039b0 msgr2=0x7f30a0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:23.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.522+0000 7f30ba000640 1 --2- 172.21.15.26:0/3376236329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30a00039b0 0x7f30a0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.522+0000 7f30ba000640 1 -- 172.21.15.26:0/3376236329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a0003520 con 0x7f30a0004800 2024-04-26T21:09:23.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.522+0000 7f30bb400640 1 -- 172.21.15.26:0/3376236329 reap_dead start 2024-04-26T21:09:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.523+0000 7f30ba000640 1 --2- 172.21.15.26:0/3376236329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a0004800 0x7f30a0004c80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f30a8007fd0 tx=0x7f30a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1afb5b92ec850ed2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.523+0000 7f30b9600640 1 -- 172.21.15.26:0/3376236329 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8007d50 con 0x7f30a0004800 2024-04-26T21:09:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.523+0000 7f30b9600640 1 -- 172.21.15.26:0/3376236329 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30a8005af0 con 0x7f30a0004800 2024-04-26T21:09:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.523+0000 7f30b9600640 1 -- 172.21.15.26:0/3376236329 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8007070 con 0x7f30a0004800 2024-04-26T21:09:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.524+0000 7f30bbe00640 1 -- 172.21.15.26:0/3376236329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a0004800 msgr2=0x7f30a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.524+0000 7f30bbe00640 1 --2- 172.21.15.26:0/3376236329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a0004800 0x7f30a0004c80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f30a8007fd0 tx=0x7f30a0005830 comp rx=0 tx=0).stop 2024-04-26T21:09:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.524+0000 7f30bbe00640 1 -- 172.21.15.26:0/3376236329 shutdown_connections 2024-04-26T21:09:23.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.524+0000 7f30bbe00640 1 --2- 172.21.15.26:0/3376236329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a0004800 0x7f30a0004c80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.524+0000 7f30bbe00640 1 -- 172.21.15.26:0/3376236329 >> 172.21.15.26:0/3376236329 conn(0x7f30bc06c690 msgr2=0x7f30bc06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:23.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.524+0000 7f30bbe00640 1 -- 172.21.15.26:0/3376236329 shutdown_connections 2024-04-26T21:09:23.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.524+0000 7f30bbe00640 1 -- 172.21.15.26:0/3376236329 wait complete. 2024-04-26T21:09:23.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.526+0000 7f30bbe00640 1 Processor -- start 2024-04-26T21:09:23.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.526+0000 7f30bbe00640 1 -- start start 2024-04-26T21:09:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.527+0000 7f30bbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a0004800 0x7f30bc13e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.527+0000 7f30bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30bc15ee40 0x7f30bc15f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.527+0000 7f30bbe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f30bc1f4ee0 0x7f30bc1f5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.527+0000 7f30bbe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f30bc146000 con 0x7f30a0004800 2024-04-26T21:09:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.527+0000 7f30bbe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f30bc0d01d0 con 0x7f30bc15ee40 2024-04-26T21:09:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.527+0000 7f30bbe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f30bc144e40 con 0x7f30bc1f4ee0 2024-04-26T21:09:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.527+0000 7f30baa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a0004800 0x7f30bc13e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.527+0000 7f30baa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a0004800 0x7f30bc13e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51042/0 (socket says 172.21.15.26:51042) 2024-04-26T21:09:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.527+0000 7f30baa00640 1 -- 172.21.15.26:0/3440583837 learned_addr learned my addr 172.21.15.26:0/3440583837 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:23.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.528+0000 7f30ba000640 1 --2- 172.21.15.26:0/3440583837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30bc15ee40 0x7f30bc15f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.528+0000 7f30baa00640 1 -- 172.21.15.26:0/3440583837 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f30bc1f4ee0 msgr2=0x7f30bc1f5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.528+0000 7f30baa00640 1 --2- 172.21.15.26:0/3440583837 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f30bc1f4ee0 0x7f30bc1f5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.528+0000 7f30baa00640 1 -- 172.21.15.26:0/3440583837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30bc15ee40 msgr2=0x7f30bc15f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.528+0000 7f30baa00640 1 --2- 172.21.15.26:0/3440583837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30bc15ee40 0x7f30bc15f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.528+0000 7f30baa00640 1 -- 172.21.15.26:0/3440583837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30bc1452d0 con 0x7f30a0004800 2024-04-26T21:09:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.528+0000 7f30ba000640 1 --2- 172.21.15.26:0/3440583837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30bc15ee40 0x7f30bc15f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.528+0000 7f30baa00640 1 --2- 172.21.15.26:0/3440583837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a0004800 0x7f30bc13e620 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009c80 tx=0x7f30ac007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.528+0000 7f30b2a00640 1 -- 172.21.15.26:0/3440583837 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac014610 con 0x7f30a0004800 2024-04-26T21:09:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.528+0000 7f30b2a00640 1 -- 172.21.15.26:0/3440583837 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30ac019070 con 0x7f30a0004800 2024-04-26T21:09:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.528+0000 7f30b2a00640 1 -- 172.21.15.26:0/3440583837 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0149f0 con 0x7f30a0004800 2024-04-26T21:09:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.528+0000 7f30bbe00640 1 -- 172.21.15.26:0/3440583837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30bc1f5c40 con 0x7f30a0004800 2024-04-26T21:09:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.529+0000 7f30bbe00640 1 -- 172.21.15.26:0/3440583837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30bc1447f0 con 0x7f30a0004800 2024-04-26T21:09:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.530+0000 7f30bbe00640 1 -- 172.21.15.26:0/3440583837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30bc06a700 con 0x7f30a0004800 2024-04-26T21:09:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.531+0000 7f30b2a00640 1 -- 172.21.15.26:0/3440583837 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30ac014b90 con 0x7f30a0004800 2024-04-26T21:09:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.532+0000 7f30b2a00640 1 --2- 172.21.15.26:0/3440583837 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f309c06c8f0 0x7f309c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.532+0000 7f30b2a00640 1 -- 172.21.15.26:0/3440583837 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f30ac08c6e0 con 0x7f30a0004800 2024-04-26T21:09:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.532+0000 7f30ba000640 1 --2- 172.21.15.26:0/3440583837 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f309c06c8f0 0x7f309c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:23.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.533+0000 7f30ba000640 1 --2- 172.21.15.26:0/3440583837 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f309c06c8f0 0x7f309c06edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f30a8033000 tx=0x7f30a802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:23.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.537+0000 7f30b2a00640 1 -- 172.21.15.26:0/3440583837 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f30ac05aa70 con 0x7f30a0004800 2024-04-26T21:09:23.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.760+0000 7f30bbe00640 1 -- 172.21.15.26:0/3440583837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f30bc070150 con 0x7f30a0004800 2024-04-26T21:09:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.761+0000 7f30b2a00640 1 -- 172.21.15.26:0/3440583837 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f30ac05a370 con 0x7f30a0004800 2024-04-26T21:09:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-04-26T21:09:23.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.764+0000 7f30bbe00640 1 -- 172.21.15.26:0/3440583837 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f309c06c8f0 msgr2=0x7f309c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:23.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.764+0000 7f30bbe00640 1 --2- 172.21.15.26:0/3440583837 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f309c06c8f0 0x7f309c06edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f30a8033000 tx=0x7f30a802e000 comp rx=0 tx=0).stop 2024-04-26T21:09:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.764+0000 7f30bbe00640 1 -- 172.21.15.26:0/3440583837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a0004800 msgr2=0x7f30bc13e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.764+0000 7f30bbe00640 1 --2- 172.21.15.26:0/3440583837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a0004800 0x7f30bc13e620 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009c80 tx=0x7f30ac007700 comp rx=0 tx=0).stop 2024-04-26T21:09:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.764+0000 7f30bbe00640 1 -- 172.21.15.26:0/3440583837 shutdown_connections 2024-04-26T21:09:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.764+0000 7f30bbe00640 1 --2- 172.21.15.26:0/3440583837 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f309c06c8f0 0x7f309c06edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.764+0000 7f30bbe00640 1 --2- 172.21.15.26:0/3440583837 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f30bc1f4ee0 0x7f30bc1f5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.764+0000 7f30bbe00640 1 --2- 172.21.15.26:0/3440583837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30bc15ee40 0x7f30bc15f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.764+0000 7f30bbe00640 1 --2- 172.21.15.26:0/3440583837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a0004800 0x7f30bc13e620 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:23.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.764+0000 7f30bbe00640 1 -- 172.21.15.26:0/3440583837 >> 172.21.15.26:0/3440583837 conn(0x7f30bc06c690 msgr2=0x7f30bc0d0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:23.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.765+0000 7f30bbe00640 1 -- 172.21.15.26:0/3440583837 shutdown_connections 2024-04-26T21:09:23.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:23.765+0000 7f30bbe00640 1 -- 172.21.15.26:0/3440583837 wait complete. 2024-04-26T21:09:23.785 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:09:23.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:23.786 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-26T21:09:23.874 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-04-26T21:09:23.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:23.875 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:09:23.933 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:09:23.933 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:09:23.989 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:09:23.990 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:09:24.045 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:09:24.046 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:09:24.099 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:09:24.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.318+0000 7f67f7c00640 1 Processor -- start 2024-04-26T21:09:24.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.318+0000 7f67f7c00640 1 -- start start 2024-04-26T21:09:24.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.319+0000 7f67f7c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f67f0070850 0x7f67f00cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:24.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.319+0000 7f67f7c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67f0070c30 0x7f67f01616f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:24.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.320+0000 7f67f7c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67f00713c0 con 0x7f67f0143360 2024-04-26T21:09:24.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.320+0000 7f67f7c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f67f00cdb20 con 0x7f67f0070850 2024-04-26T21:09:24.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.320+0000 7f67f7c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f67f0142ed0 con 0x7f67f0070c30 2024-04-26T21:09:24.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.320+0000 7f67f5c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f67f0143360 0x7f67f0071d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54592/0 (socket says 172.21.15.26:54592) 2024-04-26T21:09:24.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.320+0000 7f67f5c00640 1 -- 172.21.15.26:0/1303013233 learned_addr learned my addr 172.21.15.26:0/1303013233 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:24.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.320+0000 7f67efe00640 1 --2- 172.21.15.26:0/1303013233 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67f0070c30 0x7f67f01616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:24.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.320+0000 7f67f5200640 1 --2- 172.21.15.26:0/1303013233 >> v2:172.21.15.86:3300/0 conn(0x7f67f0070850 0x7f67f00cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:24.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.320+0000 7f67ef400640 1 -- 172.21.15.26:0/1303013233 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2881185670 0 0) 0x7f67f00713c0 con 0x7f67f0143360 2024-04-26T21:09:24.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.321+0000 7f67ef400640 1 -- 172.21.15.26:0/1303013233 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67d4003520 con 0x7f67f0143360 2024-04-26T21:09:24.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.321+0000 7f67ef400640 1 -- 172.21.15.26:0/1303013233 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2450618258 0 0) 0x7f67d4003520 con 0x7f67f0143360 2024-04-26T21:09:24.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.321+0000 7f67ef400640 1 -- 172.21.15.26:0/1303013233 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67f0070c30 msgr2=0x7f67f01616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:24.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.321+0000 7f67ef400640 1 --2- 172.21.15.26:0/1303013233 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67f0070c30 0x7f67f01616f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:24.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.321+0000 7f67ef400640 1 -- 172.21.15.26:0/1303013233 >> v2:172.21.15.86:3300/0 conn(0x7f67f0070850 msgr2=0x7f67f00cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:24.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.321+0000 7f67ef400640 1 --2- 172.21.15.26:0/1303013233 >> v2:172.21.15.86:3300/0 conn(0x7f67f0070850 0x7f67f00cf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:24.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.321+0000 7f67ef400640 1 -- 172.21.15.26:0/1303013233 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f00cd560 con 0x7f67f0143360 2024-04-26T21:09:24.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.321+0000 7f67ef400640 1 -- 172.21.15.26:0/1303013233 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f67e8001670 con 0x7f67f0143360 2024-04-26T21:09:24.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.321+0000 7f67ef400640 1 -- 172.21.15.26:0/1303013233 >> v1:172.21.15.26:6789/0 conn(0x7f67f0143360 legacy=0x7f67f0071d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:24.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.321+0000 7f67ef400640 1 --2- 172.21.15.26:0/1303013233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67d40039b0 0x7f67d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:24.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.321+0000 7f67ef400640 1 --2- 172.21.15.26:0/1303013233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d4004800 0x7f67d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:24.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.321+0000 7f67ef400640 1 --2- 172.21.15.26:0/1303013233 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67f0070c30 0x7f67d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:24.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.321+0000 7f67ef400640 1 -- 172.21.15.26:0/1303013233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f67f00cdb20 con 0x7f67d40039b0 2024-04-26T21:09:24.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.321+0000 7f67ef400640 1 -- 172.21.15.26:0/1303013233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67f0142ed0 con 0x7f67d4004800 2024-04-26T21:09:24.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.321+0000 7f67ef400640 1 -- 172.21.15.26:0/1303013233 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f67d400b9c0 con 0x7f67f0070c30 2024-04-26T21:09:24.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.321+0000 7f67f5200640 1 --2- 172.21.15.26:0/1303013233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67d40039b0 0x7f67d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:24.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.322+0000 7f67f5200640 1 -- 172.21.15.26:0/1303013233 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67f0070c30 msgr2=0x7f67d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:24.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.322+0000 7f67f5200640 1 --2- 172.21.15.26:0/1303013233 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67f0070c30 0x7f67d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:24.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.322+0000 7f67f5200640 1 -- 172.21.15.26:0/1303013233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d4004800 msgr2=0x7f67d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:24.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.322+0000 7f67f5200640 1 --2- 172.21.15.26:0/1303013233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d4004800 0x7f67d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:24.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.322+0000 7f67f5200640 1 -- 172.21.15.26:0/1303013233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d4003520 con 0x7f67d40039b0 2024-04-26T21:09:24.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.323+0000 7f67f5200640 1 --2- 172.21.15.26:0/1303013233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67d40039b0 0x7f67d4003dd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009fc0 tx=0x7f67d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffcc72518fb2a66 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:24.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.323+0000 7f67ef400640 1 -- 172.21.15.26:0/1303013233 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0009c10 con 0x7f67d40039b0 2024-04-26T21:09:24.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.323+0000 7f67ef400640 1 -- 172.21.15.26:0/1303013233 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67e0036070 con 0x7f67d40039b0 2024-04-26T21:09:24.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.323+0000 7f67ef400640 1 -- 172.21.15.26:0/1303013233 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e002d070 con 0x7f67d40039b0 2024-04-26T21:09:24.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.324+0000 7f67f7c00640 1 -- 172.21.15.26:0/1303013233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67d40039b0 msgr2=0x7f67d4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:24.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.324+0000 7f67f7c00640 1 --2- 172.21.15.26:0/1303013233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67d40039b0 0x7f67d4003dd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009fc0 tx=0x7f67d4005830 comp rx=0 tx=0).stop 2024-04-26T21:09:24.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.324+0000 7f67f5c00640 1 -- 172.21.15.26:0/1303013233 reap_dead start 2024-04-26T21:09:24.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.324+0000 7f67f7c00640 1 -- 172.21.15.26:0/1303013233 shutdown_connections 2024-04-26T21:09:24.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.324+0000 7f67f7c00640 1 -- 172.21.15.26:0/1303013233 >> 172.21.15.26:0/1303013233 conn(0x7f67f00d2f10 msgr2=0x7f67f00d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:24.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.324+0000 7f67f7c00640 1 -- 172.21.15.26:0/1303013233 shutdown_connections 2024-04-26T21:09:24.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.324+0000 7f67f7c00640 1 -- 172.21.15.26:0/1303013233 wait complete. 2024-04-26T21:09:24.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.326+0000 7f67f7c00640 1 Processor -- start 2024-04-26T21:09:24.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.326+0000 7f67f7c00640 1 -- start start 2024-04-26T21:09:24.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.327+0000 7f67f7c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67d40039b0 0x7f67f01542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:24.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.327+0000 7f67f7c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67f0154db0 0x7f67f0156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:24.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.327+0000 7f67f7c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67f00d1d50 0x7f67f0156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:24.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.327+0000 7f67f7c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f67f01437f0 con 0x7f67f0154db0 2024-04-26T21:09:24.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.327+0000 7f67f7c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67f0142b60 con 0x7f67f00d1d50 2024-04-26T21:09:24.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.327+0000 7f67f7c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f67f0143970 con 0x7f67d40039b0 2024-04-26T21:09:24.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.327+0000 7f67efe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67f0154db0 0x7f67f0156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:24.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.327+0000 7f67efe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67f0154db0 0x7f67f0156880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51062/0 (socket says 172.21.15.26:51062) 2024-04-26T21:09:24.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.327+0000 7f67efe00640 1 -- 172.21.15.26:0/502825266 learned_addr learned my addr 172.21.15.26:0/502825266 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:24.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.328+0000 7f67f5200640 1 --2- 172.21.15.26:0/502825266 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67d40039b0 0x7f67f01542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:24.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.328+0000 7f67efe00640 1 -- 172.21.15.26:0/502825266 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67d40039b0 msgr2=0x7f67f01542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:24.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.328+0000 7f67efe00640 1 --2- 172.21.15.26:0/502825266 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67d40039b0 0x7f67f01542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:24.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.328+0000 7f67efe00640 1 -- 172.21.15.26:0/502825266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67f00d1d50 msgr2=0x7f67f0156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:24.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.328+0000 7f67efe00640 1 --2- 172.21.15.26:0/502825266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67f00d1d50 0x7f67f0156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:24.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.328+0000 7f67efe00640 1 -- 172.21.15.26:0/502825266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f00b48d0 con 0x7f67f0154db0 2024-04-26T21:09:24.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.328+0000 7f67efe00640 1 --2- 172.21.15.26:0/502825266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67f0154db0 0x7f67f0156880 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f67d800e800 tx=0x7f67d800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:24.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.328+0000 7f67ecc00640 1 -- 172.21.15.26:0/502825266 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8017070 con 0x7f67f0154db0 2024-04-26T21:09:24.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.328+0000 7f67ecc00640 1 -- 172.21.15.26:0/502825266 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67d800c8e0 con 0x7f67f0154db0 2024-04-26T21:09:24.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.328+0000 7f67ecc00640 1 -- 172.21.15.26:0/502825266 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d800ccb0 con 0x7f67f0154db0 2024-04-26T21:09:24.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.328+0000 7f67f7c00640 1 -- 172.21.15.26:0/502825266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f0154860 con 0x7f67f0154db0 2024-04-26T21:09:24.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.329+0000 7f67f7c00640 1 -- 172.21.15.26:0/502825266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f00c63d0 con 0x7f67f0154db0 2024-04-26T21:09:24.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.330+0000 7f67f7c00640 1 -- 172.21.15.26:0/502825266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f00b86a0 con 0x7f67f0154db0 2024-04-26T21:09:24.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.331+0000 7f67ecc00640 1 -- 172.21.15.26:0/502825266 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f67d801a070 con 0x7f67f0154db0 2024-04-26T21:09:24.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.332+0000 7f67ecc00640 1 --2- 172.21.15.26:0/502825266 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f67cc06c8f0 0x7f67cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:24.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.332+0000 7f67ecc00640 1 -- 172.21.15.26:0/502825266 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f67d808ba80 con 0x7f67f0154db0 2024-04-26T21:09:24.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.332+0000 7f67f5200640 1 --2- 172.21.15.26:0/502825266 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f67cc06c8f0 0x7f67cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:24.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.333+0000 7f67f5200640 1 --2- 172.21.15.26:0/502825266 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f67cc06c8f0 0x7f67cc06edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f67e0035000 tx=0x7f67e002f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:24.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.336+0000 7f67ecc00640 1 -- 172.21.15.26:0/502825266 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f67f00b86a0 con 0x7f67f0154db0 2024-04-26T21:09:24.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.547+0000 7f67f7c00640 1 -- 172.21.15.26:0/502825266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67f00693d0 con 0x7f67f0154db0 2024-04-26T21:09:24.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.550+0000 7f67ecc00640 1 -- 172.21.15.26:0/502825266 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16806 (secure 0 0 0) 0x7f67d808de20 con 0x7f67f0154db0 2024-04-26T21:09:24.552 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:24.552 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":22,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:09:23.550719+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:08:35.481169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:08:36.609498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:09:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.553+0000 7f67f7c00640 1 -- 172.21.15.26:0/502825266 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f67cc06c8f0 msgr2=0x7f67cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.553+0000 7f67f7c00640 1 --2- 172.21.15.26:0/502825266 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f67cc06c8f0 0x7f67cc06edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f67e0035000 tx=0x7f67e002f000 comp rx=0 tx=0).stop 2024-04-26T21:09:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.553+0000 7f67f7c00640 1 -- 172.21.15.26:0/502825266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67f0154db0 msgr2=0x7f67f0156880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.553+0000 7f67f7c00640 1 --2- 172.21.15.26:0/502825266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67f0154db0 0x7f67f0156880 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f67d800e800 tx=0x7f67d800ecc0 comp rx=0 tx=0).stop 2024-04-26T21:09:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.553+0000 7f67f7c00640 1 -- 172.21.15.26:0/502825266 shutdown_connections 2024-04-26T21:09:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.553+0000 7f67f7c00640 1 --2- 172.21.15.26:0/502825266 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f67cc06c8f0 0x7f67cc06edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.553+0000 7f67f7c00640 1 --2- 172.21.15.26:0/502825266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67f00d1d50 0x7f67f0156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.553+0000 7f67f7c00640 1 --2- 172.21.15.26:0/502825266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67f0154db0 0x7f67f0156880 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.553+0000 7f67f7c00640 1 --2- 172.21.15.26:0/502825266 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67d40039b0 0x7f67f01542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.553+0000 7f67f7c00640 1 -- 172.21.15.26:0/502825266 >> 172.21.15.26:0/502825266 conn(0x7f67f00d2f10 msgr2=0x7f67f0071d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.553+0000 7f67f7c00640 1 -- 172.21.15.26:0/502825266 shutdown_connections 2024-04-26T21:09:24.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.553+0000 7f67f7c00640 1 -- 172.21.15.26:0/502825266 wait complete. 2024-04-26T21:09:24.576 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:08:31.322511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:08:35.481169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:08:36.609498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:09:24.577 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:09:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.772+0000 7f4b72c00640 1 Processor -- start 2024-04-26T21:09:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.772+0000 7f4b72c00640 1 -- start start 2024-04-26T21:09:24.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.774+0000 7f4b72c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4b6c0d12b0 0x7f4b6c073b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:24.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.774+0000 7f4b72c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b6c06c950 0x7f4b6c15fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:24.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.774+0000 7f4b72c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b6c160310 con 0x7f4b6c15f7b0 2024-04-26T21:09:24.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.774+0000 7f4b72c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f4b6c1605f0 con 0x7f4b6c0d12b0 2024-04-26T21:09:24.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.774+0000 7f4b72c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4b6c073370 con 0x7f4b6c06c950 2024-04-26T21:09:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.775+0000 7f4b71800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4b6c0d12b0 0x7f4b6c073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.775+0000 7f4b71800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4b6c0d12b0 0x7f4b6c073b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51070/0 (socket says 172.21.15.26:51070) 2024-04-26T21:09:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.775+0000 7f4b71800640 1 -- 172.21.15.26:0/3618575823 learned_addr learned my addr 172.21.15.26:0/3618575823 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.775+0000 7f4b72200640 1 --2- 172.21.15.26:0/3618575823 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b6c06c950 0x7f4b6c15fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.775+0000 7f4b71800640 1 -- 172.21.15.26:0/3618575823 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b6c06c950 msgr2=0x7f4b6c15fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:24.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.775+0000 7f4b71800640 1 --2- 172.21.15.26:0/3618575823 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b6c06c950 0x7f4b6c15fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:24.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.775+0000 7f4b71800640 1 -- 172.21.15.26:0/3618575823 >> v1:172.21.15.86:6789/0 conn(0x7f4b6c15f7b0 legacy=0x7f4b6c06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:24.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.775+0000 7f4b71800640 1 -- 172.21.15.26:0/3618575823 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6c073650 con 0x7f4b6c0d12b0 2024-04-26T21:09:24.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.776+0000 7f4b71800640 1 --2- 172.21.15.26:0/3618575823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b6c0d12b0 0x7f4b6c073b70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009e50 tx=0x7f4b6002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e10e9c27883d2de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:24.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.777+0000 7f4b6be00640 1 -- 172.21.15.26:0/3618575823 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b60036070 con 0x7f4b6c0d12b0 2024-04-26T21:09:24.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.777+0000 7f4b6be00640 1 -- 172.21.15.26:0/3618575823 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b60031600 con 0x7f4b6c0d12b0 2024-04-26T21:09:24.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.777+0000 7f4b6be00640 1 -- 172.21.15.26:0/3618575823 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b600319d0 con 0x7f4b6c0d12b0 2024-04-26T21:09:24.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.778+0000 7f4b72c00640 1 -- 172.21.15.26:0/3618575823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b6c0d12b0 msgr2=0x7f4b6c073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:24.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.778+0000 7f4b72c00640 1 --2- 172.21.15.26:0/3618575823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b6c0d12b0 0x7f4b6c073b70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009e50 tx=0x7f4b6002a680 comp rx=0 tx=0).stop 2024-04-26T21:09:24.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.778+0000 7f4b72c00640 1 -- 172.21.15.26:0/3618575823 shutdown_connections 2024-04-26T21:09:24.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.778+0000 7f4b72c00640 1 --2- 172.21.15.26:0/3618575823 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b6c06c950 0x7f4b6c15fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:24.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.778+0000 7f4b72c00640 1 --2- 172.21.15.26:0/3618575823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b6c0d12b0 0x7f4b6c073b70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:24.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.778+0000 7f4b72c00640 1 -- 172.21.15.26:0/3618575823 >> 172.21.15.26:0/3618575823 conn(0x7f4b6c0b3b00 msgr2=0x7f4b6c0b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:24.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.778+0000 7f4b72c00640 1 -- 172.21.15.26:0/3618575823 shutdown_connections 2024-04-26T21:09:24.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.778+0000 7f4b72c00640 1 -- 172.21.15.26:0/3618575823 wait complete. 2024-04-26T21:09:24.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.780+0000 7f4b72c00640 1 Processor -- start 2024-04-26T21:09:24.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.780+0000 7f4b72c00640 1 -- start start 2024-04-26T21:09:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.782+0000 7f4b72c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b6c06c950 0x7f4b6c0af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.782+0000 7f4b72c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b6c0d12b0 0x7f4b6c0b3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.782+0000 7f4b72c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b6c15f7b0 0x7f4b6c13fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.782+0000 7f4b72c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c0d1690 con 0x7f4b6c15f7b0 2024-04-26T21:09:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.782+0000 7f4b72c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c06f770 con 0x7f4b6c0d12b0 2024-04-26T21:09:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.782+0000 7f4b72c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4b6c1404e0 con 0x7f4b6c06c950 2024-04-26T21:09:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.782+0000 7f4b72200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b6c15f7b0 0x7f4b6c13fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.782+0000 7f4b72200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b6c15f7b0 0x7f4b6c13fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51072/0 (socket says 172.21.15.26:51072) 2024-04-26T21:09:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.782+0000 7f4b72200640 1 -- 172.21.15.26:0/1149735636 learned_addr learned my addr 172.21.15.26:0/1149735636 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.782+0000 7f4b71800640 1 --2- 172.21.15.26:0/1149735636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b6c06c950 0x7f4b6c0af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.782+0000 7f4b70e00640 1 --2- 172.21.15.26:0/1149735636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b6c0d12b0 0x7f4b6c0b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.783+0000 7f4b72200640 1 -- 172.21.15.26:0/1149735636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b6c06c950 msgr2=0x7f4b6c0af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.783+0000 7f4b72200640 1 --2- 172.21.15.26:0/1149735636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b6c06c950 0x7f4b6c0af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.783+0000 7f4b72200640 1 -- 172.21.15.26:0/1149735636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b6c0d12b0 msgr2=0x7f4b6c0b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.783+0000 7f4b72200640 1 --2- 172.21.15.26:0/1149735636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b6c0d12b0 0x7f4b6c0b3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.783+0000 7f4b72200640 1 -- 172.21.15.26:0/1149735636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6c1407c0 con 0x7f4b6c15f7b0 2024-04-26T21:09:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.783+0000 7f4b72200640 1 --2- 172.21.15.26:0/1149735636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b6c15f7b0 0x7f4b6c13fa00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400d5d0 tx=0x7f4b6400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:24.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.783+0000 7f4b69600640 1 -- 172.21.15.26:0/1149735636 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b640125f0 con 0x7f4b6c15f7b0 2024-04-26T21:09:24.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.783+0000 7f4b69600640 1 -- 172.21.15.26:0/1149735636 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b64019070 con 0x7f4b6c15f7b0 2024-04-26T21:09:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.783+0000 7f4b69600640 1 -- 172.21.15.26:0/1149735636 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b640129d0 con 0x7f4b6c15f7b0 2024-04-26T21:09:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.783+0000 7f4b72c00640 1 -- 172.21.15.26:0/1149735636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b6c140aa0 con 0x7f4b6c15f7b0 2024-04-26T21:09:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.784+0000 7f4b72c00640 1 -- 172.21.15.26:0/1149735636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b6c141060 con 0x7f4b6c15f7b0 2024-04-26T21:09:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.785+0000 7f4b72c00640 1 -- 172.21.15.26:0/1149735636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b6c0693d0 con 0x7f4b6c15f7b0 2024-04-26T21:09:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.786+0000 7f4b69600640 1 -- 172.21.15.26:0/1149735636 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b64012b70 con 0x7f4b6c15f7b0 2024-04-26T21:09:24.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.787+0000 7f4b69600640 1 --2- 172.21.15.26:0/1149735636 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4b4c06c920 0x7f4b4c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:24.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.787+0000 7f4b69600640 1 -- 172.21.15.26:0/1149735636 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f4b6408bff0 con 0x7f4b6c15f7b0 2024-04-26T21:09:24.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.787+0000 7f4b71800640 1 --2- 172.21.15.26:0/1149735636 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4b4c06c920 0x7f4b4c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:24.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.788+0000 7f4b71800640 1 --2- 172.21.15.26:0/1149735636 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4b4c06c920 0x7f4b4c06ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4b60033000 tx=0x7f4b60009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:24.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.791+0000 7f4b69600640 1 -- 172.21.15.26:0/1149735636 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4b6405a320 con 0x7f4b6c15f7b0 2024-04-26T21:09:24.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.994+0000 7f4b72c00640 1 -- 172.21.15.26:0/1149735636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4b6c141230 con 0x7f4b6c15f7b0 2024-04-26T21:09:24.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.995+0000 7f4b69600640 1 -- 172.21.15.26:0/1149735636 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f4b64059c20 con 0x7f4b6c15f7b0 2024-04-26T21:09:24.997 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-26T21:09:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.997+0000 7f4b72c00640 1 -- 172.21.15.26:0/1149735636 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4b4c06c920 msgr2=0x7f4b4c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.997+0000 7f4b72c00640 1 --2- 172.21.15.26:0/1149735636 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4b4c06c920 0x7f4b4c06ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4b60033000 tx=0x7f4b60009000 comp rx=0 tx=0).stop 2024-04-26T21:09:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.997+0000 7f4b72c00640 1 -- 172.21.15.26:0/1149735636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b6c15f7b0 msgr2=0x7f4b6c13fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.997+0000 7f4b72c00640 1 --2- 172.21.15.26:0/1149735636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b6c15f7b0 0x7f4b6c13fa00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400d5d0 tx=0x7f4b6400da90 comp rx=0 tx=0).stop 2024-04-26T21:09:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.997+0000 7f4b72c00640 1 -- 172.21.15.26:0/1149735636 shutdown_connections 2024-04-26T21:09:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.997+0000 7f4b72c00640 1 --2- 172.21.15.26:0/1149735636 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4b4c06c920 0x7f4b4c06ede0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.997+0000 7f4b72c00640 1 --2- 172.21.15.26:0/1149735636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b6c15f7b0 0x7f4b6c13fa00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.997+0000 7f4b72c00640 1 --2- 172.21.15.26:0/1149735636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b6c0d12b0 0x7f4b6c0b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.997+0000 7f4b72c00640 1 --2- 172.21.15.26:0/1149735636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b6c06c950 0x7f4b6c0af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.998+0000 7f4b72c00640 1 -- 172.21.15.26:0/1149735636 >> 172.21.15.26:0/1149735636 conn(0x7f4b6c0b3b00 msgr2=0x7f4b6c0d2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.998+0000 7f4b72c00640 1 -- 172.21.15.26:0/1149735636 shutdown_connections 2024-04-26T21:09:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:24.998+0000 7f4b72c00640 1 -- 172.21.15.26:0/1149735636 wait complete. 2024-04-26T21:09:25.019 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:09:25.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.237+0000 7f7e10200640 1 Processor -- start 2024-04-26T21:09:25.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.237+0000 7f7e10200640 1 -- start start 2024-04-26T21:09:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.238+0000 7f7e10200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7e0806f8c0 0x7f7e080b6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.238+0000 7f7e10200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7e080b6650 0x7f7e08166c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.238+0000 7f7e10200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e08146c00 con 0x7f7e080b5d30 2024-04-26T21:09:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.238+0000 7f7e10200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f7e08073700 con 0x7f7e0806f8c0 2024-04-26T21:09:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.238+0000 7f7e10200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7e0806e420 con 0x7f7e080b6650 2024-04-26T21:09:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.239+0000 7f7e0ce00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7e0806f8c0 0x7f7e080b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.239+0000 7f7e0ce00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7e0806f8c0 0x7f7e080b6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51082/0 (socket says 172.21.15.26:51082) 2024-04-26T21:09:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.239+0000 7f7e0ce00640 1 -- 172.21.15.26:0/208636615 learned_addr learned my addr 172.21.15.26:0/208636615 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.239+0000 7f7e0e200640 1 --2- 172.21.15.26:0/208636615 >> v2:172.21.15.86:3300/0 conn(0x7f7e080b6650 0x7f7e08166c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.239+0000 7f7e0ce00640 1 -- 172.21.15.26:0/208636615 >> v2:172.21.15.86:3300/0 conn(0x7f7e080b6650 msgr2=0x7f7e08166c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.239+0000 7f7e0ce00640 1 --2- 172.21.15.26:0/208636615 >> v2:172.21.15.86:3300/0 conn(0x7f7e080b6650 0x7f7e08166c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.239+0000 7f7e0ce00640 1 -- 172.21.15.26:0/208636615 >> v1:172.21.15.86:6789/0 conn(0x7f7e080b5d30 legacy=0x7f7e080b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.239+0000 7f7e0ce00640 1 -- 172.21.15.26:0/208636615 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e08072e60 con 0x7f7e0806f8c0 2024-04-26T21:09:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.240+0000 7f7e0e200640 1 --2- 172.21.15.26:0/208636615 >> v2:172.21.15.86:3300/0 conn(0x7f7e080b6650 0x7f7e08166c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.241+0000 7f7e0ce00640 1 --2- 172.21.15.26:0/208636615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e0806f8c0 0x7f7e080b6110 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7e080d0af0 tx=0x7f7df402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08e76bdeff44f37 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:25.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.241+0000 7f7e07e00640 1 -- 172.21.15.26:0/208636615 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4037070 con 0x7f7e0806f8c0 2024-04-26T21:09:25.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.241+0000 7f7e07e00640 1 -- 172.21.15.26:0/208636615 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7df4032440 con 0x7f7e0806f8c0 2024-04-26T21:09:25.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.241+0000 7f7e07e00640 1 -- 172.21.15.26:0/208636615 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4032810 con 0x7f7e0806f8c0 2024-04-26T21:09:25.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.242+0000 7f7e10200640 1 -- 172.21.15.26:0/208636615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e0806f8c0 msgr2=0x7f7e080b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.242+0000 7f7e10200640 1 --2- 172.21.15.26:0/208636615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e0806f8c0 0x7f7e080b6110 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7e080d0af0 tx=0x7f7df402a230 comp rx=0 tx=0).stop 2024-04-26T21:09:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.242+0000 7f7e10200640 1 -- 172.21.15.26:0/208636615 shutdown_connections 2024-04-26T21:09:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.242+0000 7f7e10200640 1 --2- 172.21.15.26:0/208636615 >> v2:172.21.15.86:3300/0 conn(0x7f7e080b6650 0x7f7e08166c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.242+0000 7f7e10200640 1 --2- 172.21.15.26:0/208636615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e0806f8c0 0x7f7e080b6110 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.242+0000 7f7e10200640 1 -- 172.21.15.26:0/208636615 >> 172.21.15.26:0/208636615 conn(0x7f7e080b2a80 msgr2=0x7f7e080b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.242+0000 7f7e10200640 1 -- 172.21.15.26:0/208636615 shutdown_connections 2024-04-26T21:09:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.242+0000 7f7e10200640 1 -- 172.21.15.26:0/208636615 wait complete. 2024-04-26T21:09:25.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.244+0000 7f7e10200640 1 Processor -- start 2024-04-26T21:09:25.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.244+0000 7f7e10200640 1 -- start start 2024-04-26T21:09:25.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.245+0000 7f7e10200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e0806f8c0 0x7f7e080b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:25.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.245+0000 7f7e10200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e080b5d30 0x7f7e080afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:25.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.245+0000 7f7e10200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7e080b6650 0x7f7e08159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:25.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.245+0000 7f7e10200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7e080b4490 con 0x7f7e080b5d30 2024-04-26T21:09:25.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.245+0000 7f7e10200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e08073420 con 0x7f7e0806f8c0 2024-04-26T21:09:25.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.245+0000 7f7e10200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7e08072a80 con 0x7f7e080b6650 2024-04-26T21:09:25.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.245+0000 7f7e0ce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e080b5d30 0x7f7e080afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:25.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.245+0000 7f7e0ce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e080b5d30 0x7f7e080afb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51088/0 (socket says 172.21.15.26:51088) 2024-04-26T21:09:25.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.245+0000 7f7e0ce00640 1 -- 172.21.15.26:0/4172884531 learned_addr learned my addr 172.21.15.26:0/4172884531 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:25.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.246+0000 7f7e0ce00640 1 -- 172.21.15.26:0/4172884531 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7e080b6650 msgr2=0x7f7e08159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:25.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.246+0000 7f7e0ce00640 1 --2- 172.21.15.26:0/4172884531 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7e080b6650 0x7f7e08159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.246+0000 7f7e0ce00640 1 -- 172.21.15.26:0/4172884531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e0806f8c0 msgr2=0x7f7e080b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.246+0000 7f7e0d800640 1 --2- 172.21.15.26:0/4172884531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e0806f8c0 0x7f7e080b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.246+0000 7f7e0ce00640 1 --2- 172.21.15.26:0/4172884531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e0806f8c0 0x7f7e080b1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.246+0000 7f7e0ce00640 1 -- 172.21.15.26:0/4172884531 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e080af800 con 0x7f7e080b5d30 2024-04-26T21:09:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.246+0000 7f7e0d800640 1 --2- 172.21.15.26:0/4172884531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e0806f8c0 0x7f7e080b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.246+0000 7f7e0ce00640 1 --2- 172.21.15.26:0/4172884531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e080b5d30 0x7f7e080afb10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7df4000c00 tx=0x7f7df40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.246+0000 7f7e05600640 1 -- 172.21.15.26:0/4172884531 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4041020 con 0x7f7e080b5d30 2024-04-26T21:09:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.246+0000 7f7e05600640 1 -- 172.21.15.26:0/4172884531 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7df4030970 con 0x7f7e080b5d30 2024-04-26T21:09:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.246+0000 7f7e05600640 1 -- 172.21.15.26:0/4172884531 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4037040 con 0x7f7e080b5d30 2024-04-26T21:09:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.246+0000 7f7e10200640 1 -- 172.21.15.26:0/4172884531 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e080aebf0 con 0x7f7e080b5d30 2024-04-26T21:09:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.247+0000 7f7e10200640 1 -- 172.21.15.26:0/4172884531 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e080af1b0 con 0x7f7e080b5d30 2024-04-26T21:09:25.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.248+0000 7f7e10200640 1 -- 172.21.15.26:0/4172884531 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e0806a700 con 0x7f7e080b5d30 2024-04-26T21:09:25.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.248+0000 7f7e05600640 1 -- 172.21.15.26:0/4172884531 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7df4044020 con 0x7f7e080b5d30 2024-04-26T21:09:25.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.250+0000 7f7e05600640 1 --2- 172.21.15.26:0/4172884531 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f7de806c8f0 0x7f7de806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:25.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.250+0000 7f7e05600640 1 -- 172.21.15.26:0/4172884531 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f7df40ac650 con 0x7f7e080b5d30 2024-04-26T21:09:25.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.250+0000 7f7e0d800640 1 --2- 172.21.15.26:0/4172884531 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f7de806c8f0 0x7f7de806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:25.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.251+0000 7f7e0d800640 1 --2- 172.21.15.26:0/4172884531 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f7de806c8f0 0x7f7de806edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc003eb0 tx=0x7f7dfc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:25.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.254+0000 7f7e05600640 1 -- 172.21.15.26:0/4172884531 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7df404b030 con 0x7f7e080b5d30 2024-04-26T21:09:25.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.465+0000 7f7e10200640 1 -- 172.21.15.26:0/4172884531 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7e0806d250 con 0x7f7e080b5d30 2024-04-26T21:09:25.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.466+0000 7f7e05600640 1 -- 172.21.15.26:0/4172884531 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f7df407a9f0 con 0x7f7e080b5d30 2024-04-26T21:09:25.467 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-26T21:09:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.468+0000 7f7e10200640 1 -- 172.21.15.26:0/4172884531 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f7de806c8f0 msgr2=0x7f7de806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.468+0000 7f7e10200640 1 --2- 172.21.15.26:0/4172884531 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f7de806c8f0 0x7f7de806edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc003eb0 tx=0x7f7dfc009040 comp rx=0 tx=0).stop 2024-04-26T21:09:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.468+0000 7f7e10200640 1 -- 172.21.15.26:0/4172884531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e080b5d30 msgr2=0x7f7e080afb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.468+0000 7f7e10200640 1 --2- 172.21.15.26:0/4172884531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e080b5d30 0x7f7e080afb10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7df4000c00 tx=0x7f7df40046c0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.468+0000 7f7e10200640 1 -- 172.21.15.26:0/4172884531 shutdown_connections 2024-04-26T21:09:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.468+0000 7f7e10200640 1 --2- 172.21.15.26:0/4172884531 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f7de806c8f0 0x7f7de806edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.468+0000 7f7e10200640 1 --2- 172.21.15.26:0/4172884531 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7e080b6650 0x7f7e08159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.468+0000 7f7e10200640 1 --2- 172.21.15.26:0/4172884531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e080b5d30 0x7f7e080afb10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.468+0000 7f7e10200640 1 --2- 172.21.15.26:0/4172884531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e0806f8c0 0x7f7e080b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.468+0000 7f7e10200640 1 -- 172.21.15.26:0/4172884531 >> 172.21.15.26:0/4172884531 conn(0x7f7e080b2a80 msgr2=0x7f7e080ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:25.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.469+0000 7f7e10200640 1 -- 172.21.15.26:0/4172884531 shutdown_connections 2024-04-26T21:09:25.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.469+0000 7f7e10200640 1 -- 172.21.15.26:0/4172884531 wait complete. 2024-04-26T21:09:25.490 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:09:25.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.706+0000 7f6a29a00640 1 Processor -- start 2024-04-26T21:09:25.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.706+0000 7f6a29a00640 1 -- start start 2024-04-26T21:09:25.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.707+0000 7f6a29a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6a240b74d0 0x7f6a240d0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:25.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.707+0000 7f6a29a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a240d0ab0 0x7f6a240d3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:25.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.707+0000 7f6a29a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a240b7f90 con 0x7f6a240d39c0 2024-04-26T21:09:25.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.707+0000 7f6a29a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f6a240d3df0 con 0x7f6a240d0ab0 2024-04-26T21:09:25.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.707+0000 7f6a29a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6a240b8270 con 0x7f6a240b74d0 2024-04-26T21:09:25.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.707+0000 7f6a23400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a240d0ab0 0x7f6a240d3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:25.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.708+0000 7f6a23400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a240d0ab0 0x7f6a240d3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51090/0 (socket says 172.21.15.26:51090) 2024-04-26T21:09:25.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.708+0000 7f6a23400640 1 -- 172.21.15.26:0/75268938 learned_addr learned my addr 172.21.15.26:0/75268938 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:25.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.708+0000 7f6a23e00640 1 --2- 172.21.15.26:0/75268938 >> v2:172.21.15.86:3300/0 conn(0x7f6a240b74d0 0x7f6a240d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:25.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.708+0000 7f6a23400640 1 -- 172.21.15.26:0/75268938 >> v2:172.21.15.86:3300/0 conn(0x7f6a240b74d0 msgr2=0x7f6a240d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:25.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.708+0000 7f6a23400640 1 --2- 172.21.15.26:0/75268938 >> v2:172.21.15.86:3300/0 conn(0x7f6a240b74d0 0x7f6a240d0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.708+0000 7f6a23400640 1 -- 172.21.15.26:0/75268938 >> v1:172.21.15.86:6789/0 conn(0x7f6a240d39c0 legacy=0x7f6a2415eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:25.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.708+0000 7f6a23400640 1 -- 172.21.15.26:0/75268938 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a240b8550 con 0x7f6a240d0ab0 2024-04-26T21:09:25.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.709+0000 7f6a23400640 1 --2- 172.21.15.26:0/75268938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a240d0ab0 0x7f6a240d3480 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6a2408f4f0 tx=0x7f6a0c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5baef43308c26d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:25.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.710+0000 7f6a22a00640 1 -- 172.21.15.26:0/75268938 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c037070 con 0x7f6a240d0ab0 2024-04-26T21:09:25.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.710+0000 7f6a22a00640 1 -- 172.21.15.26:0/75268938 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a0c032440 con 0x7f6a240d0ab0 2024-04-26T21:09:25.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.710+0000 7f6a22a00640 1 -- 172.21.15.26:0/75268938 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c032810 con 0x7f6a240d0ab0 2024-04-26T21:09:25.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.710+0000 7f6a29a00640 1 -- 172.21.15.26:0/75268938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a240d0ab0 msgr2=0x7f6a240d3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:25.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.710+0000 7f6a29a00640 1 --2- 172.21.15.26:0/75268938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a240d0ab0 0x7f6a240d3480 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6a2408f4f0 tx=0x7f6a0c02a230 comp rx=0 tx=0).stop 2024-04-26T21:09:25.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.710+0000 7f6a29a00640 1 -- 172.21.15.26:0/75268938 shutdown_connections 2024-04-26T21:09:25.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.710+0000 7f6a29a00640 1 --2- 172.21.15.26:0/75268938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a240d0ab0 0x7f6a240d3480 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.710+0000 7f6a29a00640 1 --2- 172.21.15.26:0/75268938 >> v2:172.21.15.86:3300/0 conn(0x7f6a240b74d0 0x7f6a240d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.710+0000 7f6a29a00640 1 -- 172.21.15.26:0/75268938 >> 172.21.15.26:0/75268938 conn(0x7f6a240d4f30 msgr2=0x7f6a240d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:25.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.710+0000 7f6a29a00640 1 -- 172.21.15.26:0/75268938 shutdown_connections 2024-04-26T21:09:25.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.710+0000 7f6a29a00640 1 -- 172.21.15.26:0/75268938 wait complete. 2024-04-26T21:09:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.712+0000 7f6a29a00640 1 Processor -- start 2024-04-26T21:09:25.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.712+0000 7f6a29a00640 1 -- start start 2024-04-26T21:09:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.713+0000 7f6a29a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a240b74d0 0x7f6a2415e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.713+0000 7f6a29a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a240d0ab0 0x7f6a241f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.713+0000 7f6a29a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a240d39c0 0x7f6a240b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.713+0000 7f6a29a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6a240b78b0 con 0x7f6a240d39c0 2024-04-26T21:09:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.713+0000 7f6a29a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6a240b8b10 con 0x7f6a240b74d0 2024-04-26T21:09:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.713+0000 7f6a29a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6a2415f290 con 0x7f6a240d0ab0 2024-04-26T21:09:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.714+0000 7f6a29000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a240d39c0 0x7f6a240b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.714+0000 7f6a29000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a240d39c0 0x7f6a240b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51106/0 (socket says 172.21.15.26:51106) 2024-04-26T21:09:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.714+0000 7f6a29000640 1 -- 172.21.15.26:0/2895495339 learned_addr learned my addr 172.21.15.26:0/2895495339 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.714+0000 7f6a23e00640 1 --2- 172.21.15.26:0/2895495339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a240b74d0 0x7f6a2415e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.714+0000 7f6a23400640 1 --2- 172.21.15.26:0/2895495339 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a240d0ab0 0x7f6a241f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.714+0000 7f6a29000640 1 -- 172.21.15.26:0/2895495339 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a240d0ab0 msgr2=0x7f6a241f50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.714+0000 7f6a29000640 1 --2- 172.21.15.26:0/2895495339 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a240d0ab0 0x7f6a241f50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.714+0000 7f6a29000640 1 -- 172.21.15.26:0/2895495339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a240b74d0 msgr2=0x7f6a2415e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.714+0000 7f6a29000640 1 --2- 172.21.15.26:0/2895495339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a240b74d0 0x7f6a2415e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.714+0000 7f6a29000640 1 -- 172.21.15.26:0/2895495339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2409edd0 con 0x7f6a240d39c0 2024-04-26T21:09:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.715+0000 7f6a29000640 1 --2- 172.21.15.26:0/2895495339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a240d39c0 0x7f6a240b5e90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00b410 tx=0x7f6a1c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.715+0000 7f6a1be00640 1 -- 172.21.15.26:0/2895495339 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c015030 con 0x7f6a240d39c0 2024-04-26T21:09:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.715+0000 7f6a1be00640 1 -- 172.21.15.26:0/2895495339 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a1c004020 con 0x7f6a240d39c0 2024-04-26T21:09:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.715+0000 7f6a1be00640 1 -- 172.21.15.26:0/2895495339 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c004410 con 0x7f6a240d39c0 2024-04-26T21:09:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.715+0000 7f6a29a00640 1 -- 172.21.15.26:0/2895495339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2409f070 con 0x7f6a240d39c0 2024-04-26T21:09:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.715+0000 7f6a29a00640 1 -- 172.21.15.26:0/2895495339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a2409f630 con 0x7f6a240d39c0 2024-04-26T21:09:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.716+0000 7f6a29a00640 1 -- 172.21.15.26:0/2895495339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a240506c0 con 0x7f6a240d39c0 2024-04-26T21:09:25.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.717+0000 7f6a1be00640 1 -- 172.21.15.26:0/2895495339 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a1c018070 con 0x7f6a240d39c0 2024-04-26T21:09:25.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.719+0000 7f6a1be00640 1 --2- 172.21.15.26:0/2895495339 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6a0006c8f0 0x7f6a0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:25.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.719+0000 7f6a1be00640 1 -- 172.21.15.26:0/2895495339 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f6a1c08b7c0 con 0x7f6a240d39c0 2024-04-26T21:09:25.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.719+0000 7f6a23e00640 1 --2- 172.21.15.26:0/2895495339 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6a0006c8f0 0x7f6a0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:25.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.720+0000 7f6a23e00640 1 --2- 172.21.15.26:0/2895495339 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6a0006c8f0 0x7f6a0006edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6a24162650 tx=0x7f6a140039b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:25.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.723+0000 7f6a1be00640 1 -- 172.21.15.26:0/2895495339 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6a1c059ad0 con 0x7f6a240d39c0 2024-04-26T21:09:25.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.927+0000 7f6a29a00640 1 -- 172.21.15.26:0/2895495339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6a24074690 con 0x7f6a240d39c0 2024-04-26T21:09:25.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.928+0000 7f6a1be00640 1 -- 172.21.15.26:0/2895495339 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f6a1c0593d0 con 0x7f6a240d39c0 2024-04-26T21:09:25.929 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-26T21:09:25.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.930+0000 7f6a29a00640 1 -- 172.21.15.26:0/2895495339 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6a0006c8f0 msgr2=0x7f6a0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:25.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.930+0000 7f6a29a00640 1 --2- 172.21.15.26:0/2895495339 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6a0006c8f0 0x7f6a0006edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6a24162650 tx=0x7f6a140039b0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.930+0000 7f6a29a00640 1 -- 172.21.15.26:0/2895495339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a240d39c0 msgr2=0x7f6a240b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:25.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.930+0000 7f6a29a00640 1 --2- 172.21.15.26:0/2895495339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a240d39c0 0x7f6a240b5e90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00b410 tx=0x7f6a1c00b8d0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.930+0000 7f6a29a00640 1 -- 172.21.15.26:0/2895495339 shutdown_connections 2024-04-26T21:09:25.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.930+0000 7f6a29a00640 1 --2- 172.21.15.26:0/2895495339 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6a0006c8f0 0x7f6a0006edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.930+0000 7f6a29a00640 1 --2- 172.21.15.26:0/2895495339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a240d39c0 0x7f6a240b5e90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.930+0000 7f6a29a00640 1 --2- 172.21.15.26:0/2895495339 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a240d0ab0 0x7f6a241f50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.930+0000 7f6a29a00640 1 --2- 172.21.15.26:0/2895495339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a240b74d0 0x7f6a2415e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:25.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.930+0000 7f6a29a00640 1 -- 172.21.15.26:0/2895495339 >> 172.21.15.26:0/2895495339 conn(0x7f6a240d4f30 msgr2=0x7f6a241600d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:25.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.931+0000 7f6a29a00640 1 -- 172.21.15.26:0/2895495339 shutdown_connections 2024-04-26T21:09:25.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:25.931+0000 7f6a29a00640 1 -- 172.21.15.26:0/2895495339 wait complete. 2024-04-26T21:09:25.950 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:09:26.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.162+0000 7f21e6600640 1 Processor -- start 2024-04-26T21:09:26.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.163+0000 7f21e6600640 1 -- start start 2024-04-26T21:09:26.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.164+0000 7f21e6600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f21e0159770 0x7f21e00b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:26.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.164+0000 7f21e6600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f21e015ffa0 0x7f21e00d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:26.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.164+0000 7f21e6600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21e0160440 con 0x7f21e0159b50 2024-04-26T21:09:26.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.164+0000 7f21e6600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f21e0160720 con 0x7f21e0159770 2024-04-26T21:09:26.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.164+0000 7f21e6600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f21e0072560 con 0x7f21e015ffa0 2024-04-26T21:09:26.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.165+0000 7f21dfe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f21e0159b50 0x7f21e015f890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54596/0 (socket says 172.21.15.26:54596) 2024-04-26T21:09:26.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.165+0000 7f21dfe00640 1 -- 172.21.15.26:0/3559358914 learned_addr learned my addr 172.21.15.26:0/3559358914 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.165+0000 7f21e5200640 1 --2- 172.21.15.26:0/3559358914 >> v2:172.21.15.86:3300/0 conn(0x7f21e0159770 0x7f21e00b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.165+0000 7f21e5c00640 1 --2- 172.21.15.26:0/3559358914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f21e015ffa0 0x7f21e00d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.165+0000 7f21df400640 1 -- 172.21.15.26:0/3559358914 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2637540416 0 0) 0x7f21e0160440 con 0x7f21e0159b50 2024-04-26T21:09:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.165+0000 7f21df400640 1 -- 172.21.15.26:0/3559358914 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21c8003520 con 0x7f21e0159b50 2024-04-26T21:09:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.166+0000 7f21df400640 1 -- 172.21.15.26:0/3559358914 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 85737247 0 0) 0x7f21c8003520 con 0x7f21e0159b50 2024-04-26T21:09:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.166+0000 7f21df400640 1 -- 172.21.15.26:0/3559358914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f21e015ffa0 msgr2=0x7f21e00d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.166+0000 7f21df400640 1 --2- 172.21.15.26:0/3559358914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f21e015ffa0 0x7f21e00d1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.166+0000 7f21df400640 1 -- 172.21.15.26:0/3559358914 >> v2:172.21.15.86:3300/0 conn(0x7f21e0159770 msgr2=0x7f21e00b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.166+0000 7f21df400640 1 --2- 172.21.15.26:0/3559358914 >> v2:172.21.15.86:3300/0 conn(0x7f21e0159770 0x7f21e00b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.166+0000 7f21df400640 1 -- 172.21.15.26:0/3559358914 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e0072250 con 0x7f21e0159b50 2024-04-26T21:09:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.166+0000 7f21df400640 1 -- 172.21.15.26:0/3559358914 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f21cc001670 con 0x7f21e0159b50 2024-04-26T21:09:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.166+0000 7f21df400640 1 -- 172.21.15.26:0/3559358914 >> v1:172.21.15.26:6789/0 conn(0x7f21e0159b50 legacy=0x7f21e015f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.166+0000 7f21df400640 1 --2- 172.21.15.26:0/3559358914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21c80039b0 0x7f21c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.166+0000 7f21df400640 1 --2- 172.21.15.26:0/3559358914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21c8004800 0x7f21c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.166+0000 7f21df400640 1 --2- 172.21.15.26:0/3559358914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f21c80051c0 0x7f21c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.166+0000 7f21df400640 1 -- 172.21.15.26:0/3559358914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f21e0160720 con 0x7f21c8004800 2024-04-26T21:09:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.166+0000 7f21df400640 1 -- 172.21.15.26:0/3559358914 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21e0072560 con 0x7f21c80039b0 2024-04-26T21:09:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.166+0000 7f21df400640 1 -- 172.21.15.26:0/3559358914 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f21c800c030 con 0x7f21c80051c0 2024-04-26T21:09:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.166+0000 7f21e5c00640 1 --2- 172.21.15.26:0/3559358914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f21e015ffa0 0x7f21e00d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:09:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.166+0000 7f21dfe00640 1 --2- 172.21.15.26:0/3559358914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21c8004800 0x7f21c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.167+0000 7f21e5200640 1 --2- 172.21.15.26:0/3559358914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21c80039b0 0x7f21c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.167+0000 7f21dfe00640 1 -- 172.21.15.26:0/3559358914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f21c80051c0 msgr2=0x7f21c800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.167+0000 7f21dfe00640 1 --2- 172.21.15.26:0/3559358914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f21c80051c0 0x7f21c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.167+0000 7f21dfe00640 1 -- 172.21.15.26:0/3559358914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21c80039b0 msgr2=0x7f21c8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.167+0000 7f21dfe00640 1 --2- 172.21.15.26:0/3559358914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21c80039b0 0x7f21c8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.167+0000 7f21dfe00640 1 -- 172.21.15.26:0/3559358914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c8003520 con 0x7f21c8004800 2024-04-26T21:09:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.168+0000 7f21dfe00640 1 --2- 172.21.15.26:0/3559358914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21c8004800 0x7f21c8004c80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00c160 tx=0x7f21c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c2e0429b1897a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.168+0000 7f21df400640 1 -- 172.21.15.26:0/3559358914 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc007c80 con 0x7f21c8004800 2024-04-26T21:09:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.168+0000 7f21df400640 1 -- 172.21.15.26:0/3559358914 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21cc005ad0 con 0x7f21c8004800 2024-04-26T21:09:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.168+0000 7f21df400640 1 -- 172.21.15.26:0/3559358914 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc007070 con 0x7f21c8004800 2024-04-26T21:09:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.169+0000 7f21e6600640 1 -- 172.21.15.26:0/3559358914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21c8004800 msgr2=0x7f21c8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.169+0000 7f21e6600640 1 --2- 172.21.15.26:0/3559358914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21c8004800 0x7f21c8004c80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00c160 tx=0x7f21c8005830 comp rx=0 tx=0).stop 2024-04-26T21:09:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.169+0000 7f21e5c00640 1 -- 172.21.15.26:0/3559358914 reap_dead start 2024-04-26T21:09:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.169+0000 7f21e6600640 1 -- 172.21.15.26:0/3559358914 shutdown_connections 2024-04-26T21:09:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.169+0000 7f21e6600640 1 -- 172.21.15.26:0/3559358914 >> 172.21.15.26:0/3559358914 conn(0x7f21e006c690 msgr2=0x7f21e006caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.169+0000 7f21e6600640 1 -- 172.21.15.26:0/3559358914 shutdown_connections 2024-04-26T21:09:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.169+0000 7f21e6600640 1 -- 172.21.15.26:0/3559358914 wait complete. 2024-04-26T21:09:26.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.171+0000 7f21e6600640 1 Processor -- start 2024-04-26T21:09:26.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.171+0000 7f21e6600640 1 -- start start 2024-04-26T21:09:26.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.172+0000 7f21e6600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f21c8004800 0x7f21e0145e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:26.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.172+0000 7f21e6600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e015eaf0 0x7f21e015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:26.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.172+0000 7f21e6600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21e015ab70 0x7f21e015aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.172+0000 7f21e6600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f21e00d1130 con 0x7f21e015ab70 2024-04-26T21:09:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.172+0000 7f21e6600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21e006ee10 con 0x7f21e015eaf0 2024-04-26T21:09:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.172+0000 7f21e6600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f21e00d12b0 con 0x7f21c8004800 2024-04-26T21:09:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.172+0000 7f21e5c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21e015ab70 0x7f21e015aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.172+0000 7f21e5c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21e015ab70 0x7f21e015aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51124/0 (socket says 172.21.15.26:51124) 2024-04-26T21:09:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.172+0000 7f21e5c00640 1 -- 172.21.15.26:0/2047560597 learned_addr learned my addr 172.21.15.26:0/2047560597 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.173+0000 7f21e5200640 1 --2- 172.21.15.26:0/2047560597 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f21c8004800 0x7f21e0145e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.173+0000 7f21e5c00640 1 -- 172.21.15.26:0/2047560597 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f21c8004800 msgr2=0x7f21e0145e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.173+0000 7f21dfe00640 1 --2- 172.21.15.26:0/2047560597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e015eaf0 0x7f21e015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.173+0000 7f21e5c00640 1 --2- 172.21.15.26:0/2047560597 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f21c8004800 0x7f21e0145e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.173+0000 7f21e5c00640 1 -- 172.21.15.26:0/2047560597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e015eaf0 msgr2=0x7f21e015ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.173+0000 7f21e5c00640 1 --2- 172.21.15.26:0/2047560597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e015eaf0 0x7f21e015ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.173+0000 7f21e5c00640 1 -- 172.21.15.26:0/2047560597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e015c130 con 0x7f21e015ab70 2024-04-26T21:09:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.173+0000 7f21e5200640 1 --2- 172.21.15.26:0/2047560597 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f21c8004800 0x7f21e0145e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.173+0000 7f21e5c00640 1 --2- 172.21.15.26:0/2047560597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21e015ab70 0x7f21e015aff0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f21e006ff90 tx=0x7f21d800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.174+0000 7f21dcc00640 1 -- 172.21.15.26:0/2047560597 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d801a070 con 0x7f21e015ab70 2024-04-26T21:09:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.174+0000 7f21dcc00640 1 -- 172.21.15.26:0/2047560597 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21d800de50 con 0x7f21e015ab70 2024-04-26T21:09:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.174+0000 7f21dcc00640 1 -- 172.21.15.26:0/2047560597 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d8003bd0 con 0x7f21e015ab70 2024-04-26T21:09:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.174+0000 7f21e6600640 1 -- 172.21.15.26:0/2047560597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e015f490 con 0x7f21e015ab70 2024-04-26T21:09:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.174+0000 7f21e6600640 1 -- 172.21.15.26:0/2047560597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e0146970 con 0x7f21e015ab70 2024-04-26T21:09:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.175+0000 7f21e6600640 1 -- 172.21.15.26:0/2047560597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21e006a700 con 0x7f21e015ab70 2024-04-26T21:09:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.176+0000 7f21dcc00640 1 -- 172.21.15.26:0/2047560597 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21d801d070 con 0x7f21e015ab70 2024-04-26T21:09:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.177+0000 7f21dcc00640 1 --2- 172.21.15.26:0/2047560597 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f21c006c920 0x7f21c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.177+0000 7f21dcc00640 1 -- 172.21.15.26:0/2047560597 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f21d808ba60 con 0x7f21e015ab70 2024-04-26T21:09:26.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.177+0000 7f21e5200640 1 --2- 172.21.15.26:0/2047560597 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f21c006c920 0x7f21c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:26.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.178+0000 7f21e5200640 1 --2- 172.21.15.26:0/2047560597 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f21c006c920 0x7f21c006ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f21d4009a80 tx=0x7f21d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:26.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.181+0000 7f21dcc00640 1 -- 172.21.15.26:0/2047560597 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f21e015c130 con 0x7f21e015ab70 2024-04-26T21:09:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.401+0000 7f21e6600640 1 -- 172.21.15.26:0/2047560597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f21e0074290 con 0x7f21e015ab70 2024-04-26T21:09:26.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.402+0000 7f21dcc00640 1 -- 172.21.15.26:0/2047560597 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f21d8010070 con 0x7f21e015ab70 2024-04-26T21:09:26.403 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-04-26T21:09:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.404+0000 7f21e6600640 1 -- 172.21.15.26:0/2047560597 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f21c006c920 msgr2=0x7f21c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.404+0000 7f21e6600640 1 --2- 172.21.15.26:0/2047560597 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f21c006c920 0x7f21c006ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f21d4009a80 tx=0x7f21d4008040 comp rx=0 tx=0).stop 2024-04-26T21:09:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.404+0000 7f21e6600640 1 -- 172.21.15.26:0/2047560597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21e015ab70 msgr2=0x7f21e015aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.404+0000 7f21e6600640 1 --2- 172.21.15.26:0/2047560597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21e015ab70 0x7f21e015aff0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f21e006ff90 tx=0x7f21d800d4a0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.404+0000 7f21e6600640 1 -- 172.21.15.26:0/2047560597 shutdown_connections 2024-04-26T21:09:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.404+0000 7f21e6600640 1 --2- 172.21.15.26:0/2047560597 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f21c006c920 0x7f21c006ede0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.404+0000 7f21e6600640 1 --2- 172.21.15.26:0/2047560597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21e015ab70 0x7f21e015aff0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.404+0000 7f21e6600640 1 --2- 172.21.15.26:0/2047560597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e015eaf0 0x7f21e015ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.404+0000 7f21e6600640 1 --2- 172.21.15.26:0/2047560597 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f21c8004800 0x7f21e0145e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.404+0000 7f21e6600640 1 -- 172.21.15.26:0/2047560597 >> 172.21.15.26:0/2047560597 conn(0x7f21e006c690 msgr2=0x7f21e006ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.404+0000 7f21e6600640 1 -- 172.21.15.26:0/2047560597 shutdown_connections 2024-04-26T21:09:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.404+0000 7f21e6600640 1 -- 172.21.15.26:0/2047560597 wait complete. 2024-04-26T21:09:26.427 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T21:09:26.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:26.427 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-26T21:09:26.527 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-26T21:09:26.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:26.527 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T21:09:26.588 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:09:26.588 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:09:26.648 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:09:26.648 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:09:26.703 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:09:26.703 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:09:26.759 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:09:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.974+0000 7fb059400640 1 Processor -- start 2024-04-26T21:09:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.974+0000 7fb059400640 1 -- start start 2024-04-26T21:09:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb059400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb054146990 0x7fb0540b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb059400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0540d13e0 0x7fb054072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb059400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0540b4480 con 0x7fb05406ec20 2024-04-26T21:09:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb059400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb054146d70 con 0x7fb054146990 2024-04-26T21:09:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb059400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb054073b70 con 0x7fb0540d13e0 2024-04-26T21:09:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb051e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb05406ec20 0x7fb0540d0cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54606/0 (socket says 172.21.15.26:54606) 2024-04-26T21:09:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb051e00640 1 -- 172.21.15.26:0/31667177 learned_addr learned my addr 172.21.15.26:0/31667177 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb052800640 1 --2- 172.21.15.26:0/31667177 >> v2:172.21.15.86:3300/0 conn(0x7fb054146990 0x7fb0540b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb053200640 1 --2- 172.21.15.26:0/31667177 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0540d13e0 0x7fb054072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb051400640 1 -- 172.21.15.26:0/31667177 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3978078379 0 0) 0x7fb0540b4480 con 0x7fb05406ec20 2024-04-26T21:09:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb051400640 1 -- 172.21.15.26:0/31667177 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb038003520 con 0x7fb05406ec20 2024-04-26T21:09:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb051400640 1 -- 172.21.15.26:0/31667177 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 752925211 0 0) 0x7fb038003520 con 0x7fb05406ec20 2024-04-26T21:09:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb051400640 1 -- 172.21.15.26:0/31667177 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0540d13e0 msgr2=0x7fb054072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb051400640 1 --2- 172.21.15.26:0/31667177 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0540d13e0 0x7fb054072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb051400640 1 -- 172.21.15.26:0/31667177 >> v2:172.21.15.86:3300/0 conn(0x7fb054146990 msgr2=0x7fb0540b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb051400640 1 --2- 172.21.15.26:0/31667177 >> v2:172.21.15.86:3300/0 conn(0x7fb054146990 0x7fb0540b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb051400640 1 -- 172.21.15.26:0/31667177 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb054073e50 con 0x7fb05406ec20 2024-04-26T21:09:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb051400640 1 -- 172.21.15.26:0/31667177 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7fb040001670 con 0x7fb05406ec20 2024-04-26T21:09:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb051400640 1 -- 172.21.15.26:0/31667177 >> v1:172.21.15.26:6789/0 conn(0x7fb05406ec20 legacy=0x7fb0540d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:26.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb051400640 1 --2- 172.21.15.26:0/31667177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0380039b0 0x7fb038003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:26.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb051400640 1 --2- 172.21.15.26:0/31667177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb038004800 0x7fb038004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:26.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb051400640 1 --2- 172.21.15.26:0/31667177 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0540d13e0 0x7fb0380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:26.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb051400640 1 -- 172.21.15.26:0/31667177 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb054146d70 con 0x7fb038004800 2024-04-26T21:09:26.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb051400640 1 -- 172.21.15.26:0/31667177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb054073b70 con 0x7fb0380039b0 2024-04-26T21:09:26.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb051400640 1 -- 172.21.15.26:0/31667177 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb03800b9c0 con 0x7fb0540d13e0 2024-04-26T21:09:26.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.977+0000 7fb051e00640 1 --2- 172.21.15.26:0/31667177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb038004800 0x7fb038004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:26.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.978+0000 7fb052800640 1 --2- 172.21.15.26:0/31667177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0380039b0 0x7fb038003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:26.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.978+0000 7fb051e00640 1 -- 172.21.15.26:0/31667177 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0540d13e0 msgr2=0x7fb0380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:26.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.978+0000 7fb051e00640 1 --2- 172.21.15.26:0/31667177 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0540d13e0 0x7fb0380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.978+0000 7fb051e00640 1 -- 172.21.15.26:0/31667177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0380039b0 msgr2=0x7fb038003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:26.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.978+0000 7fb051e00640 1 --2- 172.21.15.26:0/31667177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0380039b0 0x7fb038003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.978+0000 7fb051e00640 1 -- 172.21.15.26:0/31667177 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb038003520 con 0x7fb038004800 2024-04-26T21:09:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.979+0000 7fb051e00640 1 --2- 172.21.15.26:0/31667177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb038004800 0x7fb038004c80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb040007fd0 tx=0x7fb038005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5b2753217a7eff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.979+0000 7fb051400640 1 -- 172.21.15.26:0/31667177 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb040007d50 con 0x7fb038004800 2024-04-26T21:09:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.979+0000 7fb051400640 1 -- 172.21.15.26:0/31667177 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb040005ad0 con 0x7fb038004800 2024-04-26T21:09:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.979+0000 7fb051400640 1 -- 172.21.15.26:0/31667177 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb040007070 con 0x7fb038004800 2024-04-26T21:09:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.980+0000 7fb059400640 1 -- 172.21.15.26:0/31667177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb038004800 msgr2=0x7fb038004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.980+0000 7fb059400640 1 --2- 172.21.15.26:0/31667177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb038004800 0x7fb038004c80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb040007fd0 tx=0x7fb038005830 comp rx=0 tx=0).stop 2024-04-26T21:09:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.980+0000 7fb053200640 1 -- 172.21.15.26:0/31667177 reap_dead start 2024-04-26T21:09:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.980+0000 7fb059400640 1 -- 172.21.15.26:0/31667177 shutdown_connections 2024-04-26T21:09:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.980+0000 7fb059400640 1 -- 172.21.15.26:0/31667177 >> 172.21.15.26:0/31667177 conn(0x7fb0540cd840 msgr2=0x7fb0540cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.980+0000 7fb059400640 1 -- 172.21.15.26:0/31667177 shutdown_connections 2024-04-26T21:09:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.980+0000 7fb059400640 1 -- 172.21.15.26:0/31667177 wait complete. 2024-04-26T21:09:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.982+0000 7fb059400640 1 Processor -- start 2024-04-26T21:09:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.982+0000 7fb059400640 1 -- start start 2024-04-26T21:09:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.983+0000 7fb059400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb038004800 0x7fb054158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.983+0000 7fb059400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb054145430 0x7fb0541f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.983+0000 7fb059400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0541f65c0 0x7fb0541f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.983+0000 7fb059400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb05406e700 con 0x7fb0541f65c0 2024-04-26T21:09:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.983+0000 7fb059400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb05406e420 con 0x7fb054145430 2024-04-26T21:09:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.983+0000 7fb059400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb05406f0b0 con 0x7fb038004800 2024-04-26T21:09:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.983+0000 7fb053200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0541f65c0 0x7fb0541f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:26.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.983+0000 7fb053200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0541f65c0 0x7fb0541f2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51150/0 (socket says 172.21.15.26:51150) 2024-04-26T21:09:26.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.983+0000 7fb053200640 1 -- 172.21.15.26:0/4131842133 learned_addr learned my addr 172.21.15.26:0/4131842133 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:26.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.983+0000 7fb052800640 1 --2- 172.21.15.26:0/4131842133 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb038004800 0x7fb054158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:26.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.983+0000 7fb051e00640 1 --2- 172.21.15.26:0/4131842133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb054145430 0x7fb0541f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:26.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.983+0000 7fb053200640 1 -- 172.21.15.26:0/4131842133 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb038004800 msgr2=0x7fb054158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:26.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.983+0000 7fb053200640 1 --2- 172.21.15.26:0/4131842133 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb038004800 0x7fb054158540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.983+0000 7fb053200640 1 -- 172.21.15.26:0/4131842133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb054145430 msgr2=0x7fb0541f6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:26.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.983+0000 7fb053200640 1 --2- 172.21.15.26:0/4131842133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb054145430 0x7fb0541f6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:26.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.983+0000 7fb053200640 1 -- 172.21.15.26:0/4131842133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb054158a80 con 0x7fb0541f65c0 2024-04-26T21:09:26.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.984+0000 7fb053200640 1 --2- 172.21.15.26:0/4131842133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0541f65c0 0x7fb0541f2ab0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb0540d1d50 tx=0x7fb04c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:26.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.984+0000 7fb04aa00640 1 -- 172.21.15.26:0/4131842133 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c00bd10 con 0x7fb0541f65c0 2024-04-26T21:09:26.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.984+0000 7fb04aa00640 1 -- 172.21.15.26:0/4131842133 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb04c01a070 con 0x7fb0541f65c0 2024-04-26T21:09:26.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.984+0000 7fb04aa00640 1 -- 172.21.15.26:0/4131842133 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c014640 con 0x7fb0541f65c0 2024-04-26T21:09:26.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.984+0000 7fb059400640 1 -- 172.21.15.26:0/4131842133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb054158d10 con 0x7fb0541f65c0 2024-04-26T21:09:26.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.984+0000 7fb059400640 1 -- 172.21.15.26:0/4131842133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0541592d0 con 0x7fb0541f65c0 2024-04-26T21:09:26.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.985+0000 7fb059400640 1 -- 172.21.15.26:0/4131842133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb05406a700 con 0x7fb0541f65c0 2024-04-26T21:09:26.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.986+0000 7fb04aa00640 1 -- 172.21.15.26:0/4131842133 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb04c005d10 con 0x7fb0541f65c0 2024-04-26T21:09:26.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.987+0000 7fb04aa00640 1 --2- 172.21.15.26:0/4131842133 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb03406c920 0x7fb03406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:26.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.987+0000 7fb04aa00640 1 -- 172.21.15.26:0/4131842133 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb04c08bd10 con 0x7fb0541f65c0 2024-04-26T21:09:26.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.987+0000 7fb052800640 1 --2- 172.21.15.26:0/4131842133 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb03406c920 0x7fb03406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:26.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.988+0000 7fb052800640 1 --2- 172.21.15.26:0/4131842133 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb03406c920 0x7fb03406ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb044009910 tx=0x7fb044008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:26.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:26.991+0000 7fb04aa00640 1 -- 172.21.15.26:0/4131842133 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb04c091020 con 0x7fb0541f65c0 2024-04-26T21:09:27.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.198+0000 7fb059400640 1 -- 172.21.15.26:0/4131842133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb05406fd50 con 0x7fb0541f65c0 2024-04-26T21:09:27.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.201+0000 7fb04aa00640 1 -- 172.21.15.26:0/4131842133 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16806 (secure 0 0 0) 0x7fb04c05a020 con 0x7fb0541f65c0 2024-04-26T21:09:27.203 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:27.203 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:09:24.561115+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:08:35.481169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:08:36.609498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:09:27.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.204+0000 7fb059400640 1 -- 172.21.15.26:0/4131842133 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb03406c920 msgr2=0x7fb03406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:27.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.204+0000 7fb059400640 1 --2- 172.21.15.26:0/4131842133 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb03406c920 0x7fb03406ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb044009910 tx=0x7fb044008040 comp rx=0 tx=0).stop 2024-04-26T21:09:27.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.204+0000 7fb059400640 1 -- 172.21.15.26:0/4131842133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0541f65c0 msgr2=0x7fb0541f2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:27.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.204+0000 7fb059400640 1 --2- 172.21.15.26:0/4131842133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0541f65c0 0x7fb0541f2ab0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb0540d1d50 tx=0x7fb04c00b480 comp rx=0 tx=0).stop 2024-04-26T21:09:27.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.204+0000 7fb059400640 1 -- 172.21.15.26:0/4131842133 shutdown_connections 2024-04-26T21:09:27.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.204+0000 7fb059400640 1 --2- 172.21.15.26:0/4131842133 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb03406c920 0x7fb03406ede0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:27.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.204+0000 7fb059400640 1 --2- 172.21.15.26:0/4131842133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0541f65c0 0x7fb0541f2ab0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:27.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.204+0000 7fb059400640 1 --2- 172.21.15.26:0/4131842133 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb054145430 0x7fb0541f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:27.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.204+0000 7fb059400640 1 --2- 172.21.15.26:0/4131842133 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb038004800 0x7fb054158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:27.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.204+0000 7fb059400640 1 -- 172.21.15.26:0/4131842133 >> 172.21.15.26:0/4131842133 conn(0x7fb0540cd840 msgr2=0x7fb054156b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:27.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.204+0000 7fb059400640 1 -- 172.21.15.26:0/4131842133 shutdown_connections 2024-04-26T21:09:27.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.204+0000 7fb059400640 1 -- 172.21.15.26:0/4131842133 wait complete. 2024-04-26T21:09:27.226 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:08:31.322511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:08:35.481169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:08:36.609498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:09:27.227 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:09:27.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.446+0000 7f328f200640 1 Processor -- start 2024-04-26T21:09:27.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.446+0000 7f328f200640 1 -- start start 2024-04-26T21:09:27.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.447+0000 7f328f200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f328815bdc0 0x7f32880b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:27.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.447+0000 7f328f200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3288073530 0x7f32880cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:27.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.447+0000 7f328f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f328806fbb0 con 0x7f328806e210 2024-04-26T21:09:27.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.447+0000 7f328f200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f32880b4cf0 con 0x7f328815bdc0 2024-04-26T21:09:27.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.447+0000 7f328f200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f32880cf280 con 0x7f3288073530 2024-04-26T21:09:27.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.447+0000 7f328de00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f328815bdc0 0x7f32880b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:27.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.447+0000 7f328de00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f328815bdc0 0x7f32880b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51156/0 (socket says 172.21.15.26:51156) 2024-04-26T21:09:27.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.447+0000 7f328de00640 1 -- 172.21.15.26:0/3668784414 learned_addr learned my addr 172.21.15.26:0/3668784414 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:27.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.448+0000 7f328e800640 1 --2- 172.21.15.26:0/3668784414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3288073530 0x7f32880cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:27.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.448+0000 7f328de00640 1 -- 172.21.15.26:0/3668784414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3288073530 msgr2=0x7f32880cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:27.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.448+0000 7f328de00640 1 --2- 172.21.15.26:0/3668784414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3288073530 0x7f32880cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:27.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.448+0000 7f328de00640 1 -- 172.21.15.26:0/3668784414 >> v1:172.21.15.86:6789/0 conn(0x7f328806e210 legacy=0x7f3288072e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:27.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.448+0000 7f328de00640 1 -- 172.21.15.26:0/3668784414 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328806e780 con 0x7f328815bdc0 2024-04-26T21:09:27.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.448+0000 7f328e800640 1 --2- 172.21.15.26:0/3668784414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3288073530 0x7f32880cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:27.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.449+0000 7f328de00640 1 --2- 172.21.15.26:0/3668784414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f328815bdc0 0x7f32880b47b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f327c009870 tx=0x7f327c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5dff0472ab2d2eb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:27.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.449+0000 7f328ca00640 1 -- 172.21.15.26:0/3668784414 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c037070 con 0x7f328815bdc0 2024-04-26T21:09:27.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.449+0000 7f328ca00640 1 -- 172.21.15.26:0/3668784414 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f327c032440 con 0x7f328815bdc0 2024-04-26T21:09:27.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.449+0000 7f328ca00640 1 -- 172.21.15.26:0/3668784414 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c032810 con 0x7f328815bdc0 2024-04-26T21:09:27.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.450+0000 7f328f200640 1 -- 172.21.15.26:0/3668784414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f328815bdc0 msgr2=0x7f32880b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:27.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.450+0000 7f328f200640 1 --2- 172.21.15.26:0/3668784414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f328815bdc0 0x7f32880b47b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f327c009870 tx=0x7f327c02a180 comp rx=0 tx=0).stop 2024-04-26T21:09:27.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.450+0000 7f328f200640 1 -- 172.21.15.26:0/3668784414 shutdown_connections 2024-04-26T21:09:27.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.450+0000 7f328f200640 1 --2- 172.21.15.26:0/3668784414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3288073530 0x7f32880cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:27.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.450+0000 7f328f200640 1 --2- 172.21.15.26:0/3668784414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f328815bdc0 0x7f32880b47b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:27.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.450+0000 7f328f200640 1 -- 172.21.15.26:0/3668784414 >> 172.21.15.26:0/3668784414 conn(0x7f32880b2a10 msgr2=0x7f32880b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:27.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.450+0000 7f328f200640 1 -- 172.21.15.26:0/3668784414 shutdown_connections 2024-04-26T21:09:27.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.450+0000 7f328f200640 1 -- 172.21.15.26:0/3668784414 wait complete. 2024-04-26T21:09:27.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.452+0000 7f328f200640 1 Processor -- start 2024-04-26T21:09:27.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.452+0000 7f328f200640 1 -- start start 2024-04-26T21:09:27.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.453+0000 7f328f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328806e210 0x7f32880b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:27.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.453+0000 7f328f200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3288073530 0x7f3288149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:27.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.453+0000 7f328f200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f328815bdc0 0x7f3288149ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:27.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.453+0000 7f328f200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3288072380 con 0x7f328815bdc0 2024-04-26T21:09:27.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.453+0000 7f328f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3288070130 con 0x7f328806e210 2024-04-26T21:09:27.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.453+0000 7f328f200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f32881f8400 con 0x7f3288073530 2024-04-26T21:09:27.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.454+0000 7f328e800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f328815bdc0 0x7f3288149ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:27.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.454+0000 7f328e800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f328815bdc0 0x7f3288149ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51168/0 (socket says 172.21.15.26:51168) 2024-04-26T21:09:27.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.454+0000 7f328e800640 1 -- 172.21.15.26:0/2442381059 learned_addr learned my addr 172.21.15.26:0/2442381059 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:27.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.454+0000 7f328d400640 1 --2- 172.21.15.26:0/2442381059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3288073530 0x7f3288149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:27.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.454+0000 7f328e800640 1 -- 172.21.15.26:0/2442381059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3288073530 msgr2=0x7f3288149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:27.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.454+0000 7f328e800640 1 --2- 172.21.15.26:0/2442381059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3288073530 0x7f3288149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:27.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.454+0000 7f328de00640 1 --2- 172.21.15.26:0/2442381059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328806e210 0x7f32880b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:27.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.454+0000 7f328e800640 1 -- 172.21.15.26:0/2442381059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328806e210 msgr2=0x7f32880b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:27.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.454+0000 7f328e800640 1 --2- 172.21.15.26:0/2442381059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328806e210 0x7f32880b5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:27.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.454+0000 7f328e800640 1 -- 172.21.15.26:0/2442381059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32881f8630 con 0x7f328815bdc0 2024-04-26T21:09:27.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.455+0000 7f328e800640 1 --2- 172.21.15.26:0/2442381059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f328815bdc0 0x7f3288149ab0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f328400cef0 tx=0x7f328400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:27.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.455+0000 7f3282000640 1 -- 172.21.15.26:0/2442381059 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328401a070 con 0x7f328815bdc0 2024-04-26T21:09:27.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.455+0000 7f3282000640 1 -- 172.21.15.26:0/2442381059 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f328400d860 con 0x7f328815bdc0 2024-04-26T21:09:27.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.455+0000 7f3282000640 1 -- 172.21.15.26:0/2442381059 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328400dc50 con 0x7f328815bdc0 2024-04-26T21:09:27.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.455+0000 7f328f200640 1 -- 172.21.15.26:0/2442381059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32881f8910 con 0x7f328815bdc0 2024-04-26T21:09:27.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.455+0000 7f328f200640 1 -- 172.21.15.26:0/2442381059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32881f8ed0 con 0x7f328815bdc0 2024-04-26T21:09:27.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.456+0000 7f328f200640 1 -- 172.21.15.26:0/2442381059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32881f90d0 con 0x7f328815bdc0 2024-04-26T21:09:27.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.457+0000 7f3282000640 1 -- 172.21.15.26:0/2442381059 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f328401d070 con 0x7f328815bdc0 2024-04-26T21:09:27.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.458+0000 7f3282000640 1 --2- 172.21.15.26:0/2442381059 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f326c06c8f0 0x7f326c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:27.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.458+0000 7f3282000640 1 -- 172.21.15.26:0/2442381059 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f328408c540 con 0x7f328815bdc0 2024-04-26T21:09:27.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.459+0000 7f328de00640 1 --2- 172.21.15.26:0/2442381059 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f326c06c8f0 0x7f326c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:27.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.459+0000 7f328de00640 1 --2- 172.21.15.26:0/2442381059 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f326c06c8f0 0x7f326c06edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f327c034000 tx=0x7f327c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:27.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.463+0000 7f3282000640 1 -- 172.21.15.26:0/2442381059 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f32881f90d0 con 0x7f328815bdc0 2024-04-26T21:09:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.667+0000 7f328f200640 1 -- 172.21.15.26:0/2442381059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f328806a700 con 0x7f328815bdc0 2024-04-26T21:09:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.668+0000 7f3282000640 1 -- 172.21.15.26:0/2442381059 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f328405a910 con 0x7f328815bdc0 2024-04-26T21:09:27.670 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-26T21:09:27.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.671+0000 7f328f200640 1 -- 172.21.15.26:0/2442381059 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f326c06c8f0 msgr2=0x7f326c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:27.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.671+0000 7f328f200640 1 --2- 172.21.15.26:0/2442381059 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f326c06c8f0 0x7f326c06edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f327c034000 tx=0x7f327c02e000 comp rx=0 tx=0).stop 2024-04-26T21:09:27.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.671+0000 7f328f200640 1 -- 172.21.15.26:0/2442381059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f328815bdc0 msgr2=0x7f3288149ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:27.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.671+0000 7f328f200640 1 --2- 172.21.15.26:0/2442381059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f328815bdc0 0x7f3288149ab0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f328400cef0 tx=0x7f328400a710 comp rx=0 tx=0).stop 2024-04-26T21:09:27.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.671+0000 7f328f200640 1 -- 172.21.15.26:0/2442381059 shutdown_connections 2024-04-26T21:09:27.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.671+0000 7f328f200640 1 --2- 172.21.15.26:0/2442381059 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f326c06c8f0 0x7f326c06edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:27.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.671+0000 7f328f200640 1 --2- 172.21.15.26:0/2442381059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f328815bdc0 0x7f3288149ab0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:27.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.671+0000 7f328f200640 1 --2- 172.21.15.26:0/2442381059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3288073530 0x7f3288149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:27.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.671+0000 7f328f200640 1 --2- 172.21.15.26:0/2442381059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328806e210 0x7f32880b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:27.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.671+0000 7f328f200640 1 -- 172.21.15.26:0/2442381059 >> 172.21.15.26:0/2442381059 conn(0x7f32880b2a10 msgr2=0x7f32880b6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:27.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.671+0000 7f328f200640 1 -- 172.21.15.26:0/2442381059 shutdown_connections 2024-04-26T21:09:27.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.671+0000 7f328f200640 1 -- 172.21.15.26:0/2442381059 wait complete. 2024-04-26T21:09:27.691 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:09:27.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.896+0000 7fa53b200640 1 Processor -- start 2024-04-26T21:09:27.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.896+0000 7fa53b200640 1 -- start start 2024-04-26T21:09:27.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.898+0000 7fa53b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5340b5d30 0x7fa5340b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:27.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.898+0000 7fa53b200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa5340b42c0 0x7fa5340b6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:27.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.898+0000 7fa53b200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa534146c00 con 0x7fa5340b6650 2024-04-26T21:09:27.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.898+0000 7fa53b200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fa534073700 con 0x7fa5340b42c0 2024-04-26T21:09:27.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.898+0000 7fa53b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa53406e420 con 0x7fa5340b5d30 2024-04-26T21:09:27.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.898+0000 7fa533400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa5340b42c0 0x7fa5340b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:27.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.898+0000 7fa533400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa5340b42c0 0x7fa5340b6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51174/0 (socket says 172.21.15.26:51174) 2024-04-26T21:09:27.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.898+0000 7fa533400640 1 -- 172.21.15.26:0/741566301 learned_addr learned my addr 172.21.15.26:0/741566301 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:27.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.898+0000 7fa533e00640 1 --2- 172.21.15.26:0/741566301 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5340b5d30 0x7fa5340b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:27.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.898+0000 7fa533400640 1 -- 172.21.15.26:0/741566301 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5340b5d30 msgr2=0x7fa5340b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:27.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.898+0000 7fa533400640 1 --2- 172.21.15.26:0/741566301 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5340b5d30 0x7fa5340b3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:27.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.898+0000 7fa533400640 1 -- 172.21.15.26:0/741566301 >> v1:172.21.15.26:6789/0 conn(0x7fa5340b6650 legacy=0x7fa53406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:27.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.898+0000 7fa533400640 1 -- 172.21.15.26:0/741566301 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa534072e60 con 0x7fa5340b42c0 2024-04-26T21:09:27.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.898+0000 7fa533e00640 1 --2- 172.21.15.26:0/741566301 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5340b5d30 0x7fa5340b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:27.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.900+0000 7fa533400640 1 --2- 172.21.15.26:0/741566301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5340b42c0 0x7fa5340b6110 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa520009870 tx=0x7fa52002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca6e26e0e1e199d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:27.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.900+0000 7fa532a00640 1 -- 172.21.15.26:0/741566301 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520037070 con 0x7fa5340b42c0 2024-04-26T21:09:27.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.900+0000 7fa532a00640 1 -- 172.21.15.26:0/741566301 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa520032440 con 0x7fa5340b42c0 2024-04-26T21:09:27.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.900+0000 7fa532a00640 1 -- 172.21.15.26:0/741566301 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520032810 con 0x7fa5340b42c0 2024-04-26T21:09:27.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.900+0000 7fa53b200640 1 -- 172.21.15.26:0/741566301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5340b42c0 msgr2=0x7fa5340b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:27.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.900+0000 7fa53b200640 1 --2- 172.21.15.26:0/741566301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5340b42c0 0x7fa5340b6110 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa520009870 tx=0x7fa52002a210 comp rx=0 tx=0).stop 2024-04-26T21:09:27.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.901+0000 7fa53b200640 1 -- 172.21.15.26:0/741566301 shutdown_connections 2024-04-26T21:09:27.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.901+0000 7fa53b200640 1 --2- 172.21.15.26:0/741566301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5340b42c0 0x7fa5340b6110 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:27.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.901+0000 7fa53b200640 1 --2- 172.21.15.26:0/741566301 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5340b5d30 0x7fa5340b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:27.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.901+0000 7fa53b200640 1 -- 172.21.15.26:0/741566301 >> 172.21.15.26:0/741566301 conn(0x7fa5340b2a80 msgr2=0x7fa5340b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:27.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.901+0000 7fa53b200640 1 -- 172.21.15.26:0/741566301 shutdown_connections 2024-04-26T21:09:27.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.901+0000 7fa53b200640 1 -- 172.21.15.26:0/741566301 wait complete. 2024-04-26T21:09:27.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.902+0000 7fa53b200640 1 Processor -- start 2024-04-26T21:09:27.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.903+0000 7fa53b200640 1 -- start start 2024-04-26T21:09:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.904+0000 7fa53b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5340b42c0 0x7fa5340b1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.904+0000 7fa53b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5340b5d30 0x7fa5341f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.904+0000 7fa53b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5340b6650 0x7fa5341e6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.904+0000 7fa53b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa534072a80 con 0x7fa5340b6650 2024-04-26T21:09:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.904+0000 7fa53b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa534073420 con 0x7fa5340b42c0 2024-04-26T21:09:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.904+0000 7fa53b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa534072c00 con 0x7fa5340b5d30 2024-04-26T21:09:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.904+0000 7fa539200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5340b6650 0x7fa5341e6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.904+0000 7fa539200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5340b6650 0x7fa5341e6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51188/0 (socket says 172.21.15.26:51188) 2024-04-26T21:09:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.904+0000 7fa539200640 1 -- 172.21.15.26:0/611760859 learned_addr learned my addr 172.21.15.26:0/611760859 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.904+0000 7fa533e00640 1 --2- 172.21.15.26:0/611760859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5340b42c0 0x7fa5340b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.904+0000 7fa533400640 1 --2- 172.21.15.26:0/611760859 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5340b5d30 0x7fa5341f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.904+0000 7fa539200640 1 -- 172.21.15.26:0/611760859 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5340b5d30 msgr2=0x7fa5341f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.904+0000 7fa539200640 1 --2- 172.21.15.26:0/611760859 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5340b5d30 0x7fa5341f5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.904+0000 7fa539200640 1 -- 172.21.15.26:0/611760859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5340b42c0 msgr2=0x7fa5340b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.904+0000 7fa539200640 1 --2- 172.21.15.26:0/611760859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5340b42c0 0x7fa5340b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.904+0000 7fa539200640 1 -- 172.21.15.26:0/611760859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5341f6030 con 0x7fa5340b6650 2024-04-26T21:09:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.904+0000 7fa533e00640 1 --2- 172.21.15.26:0/611760859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5340b42c0 0x7fa5340b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.905+0000 7fa539200640 1 --2- 172.21.15.26:0/611760859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5340b6650 0x7fa5341e6c00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00c780 tx=0x7fa52c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.905+0000 7fa52be00640 1 -- 172.21.15.26:0/611760859 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c016020 con 0x7fa5340b6650 2024-04-26T21:09:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.905+0000 7fa52be00640 1 -- 172.21.15.26:0/611760859 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa52c00aa70 con 0x7fa5340b6650 2024-04-26T21:09:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.905+0000 7fa52be00640 1 -- 172.21.15.26:0/611760859 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c00ce30 con 0x7fa5340b6650 2024-04-26T21:09:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.905+0000 7fa53b200640 1 -- 172.21.15.26:0/611760859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5340b1d90 con 0x7fa5340b6650 2024-04-26T21:09:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.905+0000 7fa53b200640 1 -- 172.21.15.26:0/611760859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5340b2350 con 0x7fa5340b6650 2024-04-26T21:09:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.906+0000 7fa53b200640 1 -- 172.21.15.26:0/611760859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa53406a700 con 0x7fa5340b6650 2024-04-26T21:09:27.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.908+0000 7fa52be00640 1 -- 172.21.15.26:0/611760859 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa52c019020 con 0x7fa5340b6650 2024-04-26T21:09:27.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.910+0000 7fa52be00640 1 --2- 172.21.15.26:0/611760859 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa51406c8f0 0x7fa51406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:27.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.910+0000 7fa52be00640 1 -- 172.21.15.26:0/611760859 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fa52c08bab0 con 0x7fa5340b6650 2024-04-26T21:09:27.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.910+0000 7fa533e00640 1 --2- 172.21.15.26:0/611760859 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa51406c8f0 0x7fa51406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:27.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.910+0000 7fa533e00640 1 --2- 172.21.15.26:0/611760859 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa51406c8f0 0x7fa51406edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa524006fd0 tx=0x7fa524008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:27.915+0000 7fa52be00640 1 -- 172.21.15.26:0/611760859 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa52c020030 con 0x7fa5340b6650 2024-04-26T21:09:28.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.119+0000 7fa53b200640 1 -- 172.21.15.26:0/611760859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa534072750 con 0x7fa5340b6650 2024-04-26T21:09:28.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.121+0000 7fa52be00640 1 -- 172.21.15.26:0/611760859 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7fa52c059e50 con 0x7fa5340b6650 2024-04-26T21:09:28.122 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-26T21:09:28.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.123+0000 7fa53b200640 1 -- 172.21.15.26:0/611760859 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa51406c8f0 msgr2=0x7fa51406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:28.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.123+0000 7fa53b200640 1 --2- 172.21.15.26:0/611760859 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa51406c8f0 0x7fa51406edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa524006fd0 tx=0x7fa524008040 comp rx=0 tx=0).stop 2024-04-26T21:09:28.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.123+0000 7fa53b200640 1 -- 172.21.15.26:0/611760859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5340b6650 msgr2=0x7fa5341e6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:28.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.123+0000 7fa53b200640 1 --2- 172.21.15.26:0/611760859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5340b6650 0x7fa5341e6c00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00c780 tx=0x7fa52c00cc40 comp rx=0 tx=0).stop 2024-04-26T21:09:28.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.123+0000 7fa53b200640 1 -- 172.21.15.26:0/611760859 shutdown_connections 2024-04-26T21:09:28.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.123+0000 7fa53b200640 1 --2- 172.21.15.26:0/611760859 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa51406c8f0 0x7fa51406edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.123+0000 7fa53b200640 1 --2- 172.21.15.26:0/611760859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5340b6650 0x7fa5341e6c00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.123+0000 7fa53b200640 1 --2- 172.21.15.26:0/611760859 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5340b5d30 0x7fa5341f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.123+0000 7fa53b200640 1 --2- 172.21.15.26:0/611760859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5340b42c0 0x7fa5340b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.123+0000 7fa53b200640 1 -- 172.21.15.26:0/611760859 >> 172.21.15.26:0/611760859 conn(0x7fa5340b2a80 msgr2=0x7fa53406e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:28.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.123+0000 7fa53b200640 1 -- 172.21.15.26:0/611760859 shutdown_connections 2024-04-26T21:09:28.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.123+0000 7fa53b200640 1 -- 172.21.15.26:0/611760859 wait complete. 2024-04-26T21:09:28.144 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:09:28.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.361+0000 7ff8f1000640 1 Processor -- start 2024-04-26T21:09:28.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.361+0000 7ff8f1000640 1 -- start start 2024-04-26T21:09:28.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.363+0000 7ff8f1000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff8ec06f9f0 0x7ff8ec0cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:28.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.363+0000 7ff8f1000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8ec0cebd0 con 0x7ff8ec15d590 2024-04-26T21:09:28.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.363+0000 7ff8f1000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8ec0ceeb0 con 0x7ff8ec146a30 2024-04-26T21:09:28.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.363+0000 7ff8f1000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff8ec0cf190 con 0x7ff8ec06f9f0 2024-04-26T21:09:28.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.363+0000 7ff8eaa00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff8ec15d590 0x7ff8ec06f2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54616/0 (socket says 172.21.15.26:54616) 2024-04-26T21:09:28.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.363+0000 7ff8eaa00640 1 -- 172.21.15.26:0/2761506762 learned_addr learned my addr 172.21.15.26:0/2761506762 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:28.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.364+0000 7ff8ea000640 1 -- 172.21.15.26:0/2761506762 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 571383365 0 0) 0x7ff8ec0cebd0 con 0x7ff8ec15d590 2024-04-26T21:09:28.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.364+0000 7ff8ea000640 1 -- 172.21.15.26:0/2761506762 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8d0003520 con 0x7ff8ec15d590 2024-04-26T21:09:28.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.364+0000 7ff8ebe00640 1 --2- 172.21.15.26:0/2761506762 >> v2:172.21.15.86:3300/0 conn(0x7ff8ec06f9f0 0x7ff8ec0cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:28.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.364+0000 7ff8ea000640 1 -- 172.21.15.26:0/2761506762 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2252474313 0 0) 0x7ff8d0003520 con 0x7ff8ec15d590 2024-04-26T21:09:28.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.364+0000 7ff8ea000640 1 -- 172.21.15.26:0/2761506762 >> v2:172.21.15.86:3300/0 conn(0x7ff8ec06f9f0 msgr2=0x7ff8ec0cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:28.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.364+0000 7ff8ea000640 1 --2- 172.21.15.26:0/2761506762 >> v2:172.21.15.86:3300/0 conn(0x7ff8ec06f9f0 0x7ff8ec0cff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.364+0000 7ff8ea000640 1 -- 172.21.15.26:0/2761506762 >> v1:172.21.15.86:6789/0 conn(0x7ff8ec146a30 legacy=0x7ff8ec0b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:28.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.364+0000 7ff8ea000640 1 -- 172.21.15.26:0/2761506762 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ec0cf470 con 0x7ff8ec15d590 2024-04-26T21:09:28.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.364+0000 7ff8ea000640 1 -- 172.21.15.26:0/2761506762 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7ff8d8001690 con 0x7ff8ec15d590 2024-04-26T21:09:28.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.364+0000 7ff8ea000640 1 -- 172.21.15.26:0/2761506762 >> v1:172.21.15.26:6789/0 conn(0x7ff8ec15d590 legacy=0x7ff8ec06f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:28.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.364+0000 7ff8ea000640 1 --2- 172.21.15.26:0/2761506762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8d0003a00 0x7ff8d0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:28.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.364+0000 7ff8ea000640 1 --2- 172.21.15.26:0/2761506762 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8d0004800 0x7ff8d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:28.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.364+0000 7ff8ea000640 1 --2- 172.21.15.26:0/2761506762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8d0005210 0x7ff8d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:28.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.364+0000 7ff8ea000640 1 -- 172.21.15.26:0/2761506762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec0cf190 con 0x7ff8d0003a00 2024-04-26T21:09:28.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.364+0000 7ff8ea000640 1 -- 172.21.15.26:0/2761506762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d000c160 con 0x7ff8d0005210 2024-04-26T21:09:28.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.364+0000 7ff8ea000640 1 -- 172.21.15.26:0/2761506762 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff8d000c360 con 0x7ff8d0004800 2024-04-26T21:09:28.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.365+0000 7ff8eb400640 1 --2- 172.21.15.26:0/2761506762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8d0003a00 0x7ff8d0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:28.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.365+0000 7ff8eaa00640 1 --2- 172.21.15.26:0/2761506762 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8d0004800 0x7ff8d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:28.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.365+0000 7ff8eb400640 1 -- 172.21.15.26:0/2761506762 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8d0004800 msgr2=0x7ff8d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:28.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.365+0000 7ff8eb400640 1 --2- 172.21.15.26:0/2761506762 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8d0004800 0x7ff8d0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.365+0000 7ff8eb400640 1 -- 172.21.15.26:0/2761506762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8d0005210 msgr2=0x7ff8d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:28.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.365+0000 7ff8ebe00640 1 --2- 172.21.15.26:0/2761506762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8d0005210 0x7ff8d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:28.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.365+0000 7ff8eb400640 1 --2- 172.21.15.26:0/2761506762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8d0005210 0x7ff8d000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.365+0000 7ff8eb400640 1 -- 172.21.15.26:0/2761506762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d0003520 con 0x7ff8d0003a00 2024-04-26T21:09:28.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.365+0000 7ff8ebe00640 1 -- 172.21.15.26:0/2761506762 reap_dead start 2024-04-26T21:09:28.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.366+0000 7ff8eb400640 1 --2- 172.21.15.26:0/2761506762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8d0003a00 0x7ff8d0003e20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc001770 tx=0x7ff8d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d0dec5ce617b68 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:28.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.367+0000 7ff8ea000640 1 -- 172.21.15.26:0/2761506762 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0013c0 con 0x7ff8d0003a00 2024-04-26T21:09:28.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.367+0000 7ff8ea000640 1 -- 172.21.15.26:0/2761506762 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8dc0052d0 con 0x7ff8d0003a00 2024-04-26T21:09:28.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.367+0000 7ff8ea000640 1 -- 172.21.15.26:0/2761506762 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc02a8a0 con 0x7ff8d0003a00 2024-04-26T21:09:28.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.367+0000 7ff8f1000640 1 -- 172.21.15.26:0/2761506762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8d0003a00 msgr2=0x7ff8d0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:28.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.367+0000 7ff8f1000640 1 --2- 172.21.15.26:0/2761506762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8d0003a00 0x7ff8d0003e20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc001770 tx=0x7ff8d0005830 comp rx=0 tx=0).stop 2024-04-26T21:09:28.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.367+0000 7ff8f1000640 1 -- 172.21.15.26:0/2761506762 shutdown_connections 2024-04-26T21:09:28.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.367+0000 7ff8f1000640 1 --2- 172.21.15.26:0/2761506762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8d0003a00 0x7ff8d0003e20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.367+0000 7ff8f1000640 1 -- 172.21.15.26:0/2761506762 >> 172.21.15.26:0/2761506762 conn(0x7ff8ec0cd7d0 msgr2=0x7ff8ec0cdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:28.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.367+0000 7ff8f1000640 1 -- 172.21.15.26:0/2761506762 shutdown_connections 2024-04-26T21:09:28.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.368+0000 7ff8f1000640 1 -- 172.21.15.26:0/2761506762 wait complete. 2024-04-26T21:09:28.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.369+0000 7ff8f1000640 1 Processor -- start 2024-04-26T21:09:28.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.369+0000 7ff8f1000640 1 -- start start 2024-04-26T21:09:28.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.370+0000 7ff8f1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8d0003a00 0x7ff8ec15c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:28.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.370+0000 7ff8f1000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8ec1f4ef0 0x7ff8ec144fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:28.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.370+0000 7ff8f1000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8ec1454f0 0x7ff8ec1e6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:28.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.370+0000 7ff8f1000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec0b6d20 con 0x7ff8ec1454f0 2024-04-26T21:09:28.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.370+0000 7ff8f1000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec0cfa30 con 0x7ff8d0003a00 2024-04-26T21:09:28.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.370+0000 7ff8f1000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff8ec15da20 con 0x7ff8ec1f4ef0 2024-04-26T21:09:28.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.371+0000 7ff8ebe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8ec1454f0 0x7ff8ec1e6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:28.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.371+0000 7ff8ebe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8ec1454f0 0x7ff8ec1e6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51208/0 (socket says 172.21.15.26:51208) 2024-04-26T21:09:28.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.371+0000 7ff8ebe00640 1 -- 172.21.15.26:0/3339525590 learned_addr learned my addr 172.21.15.26:0/3339525590 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:28.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.371+0000 7ff8eaa00640 1 --2- 172.21.15.26:0/3339525590 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8ec1f4ef0 0x7ff8ec144fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:28.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.371+0000 7ff8eb400640 1 --2- 172.21.15.26:0/3339525590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8d0003a00 0x7ff8ec15c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:28.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.371+0000 7ff8ebe00640 1 -- 172.21.15.26:0/3339525590 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8ec1f4ef0 msgr2=0x7ff8ec144fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:28.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.371+0000 7ff8ebe00640 1 --2- 172.21.15.26:0/3339525590 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8ec1f4ef0 0x7ff8ec144fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.371+0000 7ff8ebe00640 1 -- 172.21.15.26:0/3339525590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8d0003a00 msgr2=0x7ff8ec15c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:28.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.371+0000 7ff8ebe00640 1 --2- 172.21.15.26:0/3339525590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8d0003a00 0x7ff8ec15c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.371+0000 7ff8ebe00640 1 -- 172.21.15.26:0/3339525590 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ec1e7960 con 0x7ff8ec1454f0 2024-04-26T21:09:28.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.371+0000 7ff8eb400640 1 --2- 172.21.15.26:0/3339525590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8d0003a00 0x7ff8ec15c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:28.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.372+0000 7ff8ebe00640 1 --2- 172.21.15.26:0/3339525590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8ec1454f0 0x7ff8ec1e6950 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec0b3a80 tx=0x7ff8e4009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:28.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.372+0000 7ff8e3400640 1 -- 172.21.15.26:0/3339525590 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4019070 con 0x7ff8ec1454f0 2024-04-26T21:09:28.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.372+0000 7ff8e3400640 1 -- 172.21.15.26:0/3339525590 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e4007cb0 con 0x7ff8ec1454f0 2024-04-26T21:09:28.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.372+0000 7ff8e3400640 1 -- 172.21.15.26:0/3339525590 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4007e60 con 0x7ff8ec1454f0 2024-04-26T21:09:28.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.372+0000 7ff8f1000640 1 -- 172.21.15.26:0/3339525590 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ec1e7680 con 0x7ff8ec1454f0 2024-04-26T21:09:28.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.372+0000 7ff8f1000640 1 -- 172.21.15.26:0/3339525590 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ec143200 con 0x7ff8ec1454f0 2024-04-26T21:09:28.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.373+0000 7ff8f1000640 1 -- 172.21.15.26:0/3339525590 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8ec0506c0 con 0x7ff8ec1454f0 2024-04-26T21:09:28.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.374+0000 7ff8e3400640 1 -- 172.21.15.26:0/3339525590 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8e4005d10 con 0x7ff8ec1454f0 2024-04-26T21:09:28.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.375+0000 7ff8e3400640 1 --2- 172.21.15.26:0/3339525590 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff8cc06c8f0 0x7ff8cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:28.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.375+0000 7ff8e3400640 1 -- 172.21.15.26:0/3339525590 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7ff8e408b6f0 con 0x7ff8ec1454f0 2024-04-26T21:09:28.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.375+0000 7ff8eb400640 1 --2- 172.21.15.26:0/3339525590 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff8cc06c8f0 0x7ff8cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:28.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.376+0000 7ff8eb400640 1 --2- 172.21.15.26:0/3339525590 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff8cc06c8f0 0x7ff8cc06edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc032000 tx=0x7ff8dc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:28.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.379+0000 7ff8e3400640 1 -- 172.21.15.26:0/3339525590 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff8e4059a90 con 0x7ff8ec1454f0 2024-04-26T21:09:28.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.592+0000 7ff8f1000640 1 -- 172.21.15.26:0/3339525590 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff8ec0739a0 con 0x7ff8ec1454f0 2024-04-26T21:09:28.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.593+0000 7ff8e3400640 1 -- 172.21.15.26:0/3339525590 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7ff8e4059390 con 0x7ff8ec1454f0 2024-04-26T21:09:28.593 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-26T21:09:28.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.595+0000 7ff8f1000640 1 -- 172.21.15.26:0/3339525590 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff8cc06c8f0 msgr2=0x7ff8cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:28.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.595+0000 7ff8f1000640 1 --2- 172.21.15.26:0/3339525590 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff8cc06c8f0 0x7ff8cc06edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc032000 tx=0x7ff8dc02e000 comp rx=0 tx=0).stop 2024-04-26T21:09:28.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.595+0000 7ff8f1000640 1 -- 172.21.15.26:0/3339525590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8ec1454f0 msgr2=0x7ff8ec1e6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:28.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.595+0000 7ff8f1000640 1 --2- 172.21.15.26:0/3339525590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8ec1454f0 0x7ff8ec1e6950 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec0b3a80 tx=0x7ff8e4009cd0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.595+0000 7ff8f1000640 1 -- 172.21.15.26:0/3339525590 shutdown_connections 2024-04-26T21:09:28.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.595+0000 7ff8f1000640 1 --2- 172.21.15.26:0/3339525590 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff8cc06c8f0 0x7ff8cc06edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.595+0000 7ff8f1000640 1 --2- 172.21.15.26:0/3339525590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8ec1454f0 0x7ff8ec1e6950 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.595+0000 7ff8f1000640 1 --2- 172.21.15.26:0/3339525590 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8ec1f4ef0 0x7ff8ec144fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.595+0000 7ff8f1000640 1 --2- 172.21.15.26:0/3339525590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8d0003a00 0x7ff8ec15c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.595+0000 7ff8f1000640 1 -- 172.21.15.26:0/3339525590 >> 172.21.15.26:0/3339525590 conn(0x7ff8ec0cd7d0 msgr2=0x7ff8ec0ae850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:28.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.596+0000 7ff8f1000640 1 -- 172.21.15.26:0/3339525590 shutdown_connections 2024-04-26T21:09:28.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.596+0000 7ff8f1000640 1 -- 172.21.15.26:0/3339525590 wait complete. 2024-04-26T21:09:28.616 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:09:28.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.818+0000 7f6a85400640 1 Processor -- start 2024-04-26T21:09:28.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.819+0000 7f6a85400640 1 -- start start 2024-04-26T21:09:28.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.820+0000 7f6a85400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a800d3cf0 0x7f6a800cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:28.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.820+0000 7f6a85400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6a8015f4d0 0x7f6a801638f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:28.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.820+0000 7f6a85400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a800d02b0 con 0x7f6a8015f0f0 2024-04-26T21:09:28.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.821+0000 7f6a85400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f6a800baf40 con 0x7f6a800d3cf0 2024-04-26T21:09:28.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.821+0000 7f6a85400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6a80163e30 con 0x7f6a8015f4d0 2024-04-26T21:09:28.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.821+0000 7f6a7fe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6a8015f0f0 0x7f6a800d35e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54632/0 (socket says 172.21.15.26:54632) 2024-04-26T21:09:28.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.821+0000 7f6a7f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a800d3cf0 0x7f6a800cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:28.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.821+0000 7f6a7fe00640 1 -- 172.21.15.26:0/918828615 learned_addr learned my addr 172.21.15.26:0/918828615 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:28.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.821+0000 7f6a84a00640 1 --2- 172.21.15.26:0/918828615 >> v2:172.21.15.86:3300/0 conn(0x7f6a8015f4d0 0x7f6a801638f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:28.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.821+0000 7f6a7ea00640 1 -- 172.21.15.26:0/918828615 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1463050541 0 0) 0x7f6a800d02b0 con 0x7f6a8015f0f0 2024-04-26T21:09:28.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.821+0000 7f6a7f400640 1 -- 172.21.15.26:0/918828615 >> v2:172.21.15.86:3300/0 conn(0x7f6a8015f4d0 msgr2=0x7f6a801638f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:28.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.821+0000 7f6a7f400640 1 --2- 172.21.15.26:0/918828615 >> v2:172.21.15.86:3300/0 conn(0x7f6a8015f4d0 0x7f6a801638f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.821+0000 7f6a7f400640 1 -- 172.21.15.26:0/918828615 >> v1:172.21.15.26:6789/0 conn(0x7f6a8015f0f0 legacy=0x7f6a800d35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:28.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.821+0000 7f6a7f400640 1 -- 172.21.15.26:0/918828615 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a800d0590 con 0x7f6a800d3cf0 2024-04-26T21:09:28.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.823+0000 7f6a7f400640 1 --2- 172.21.15.26:0/918828615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a800d3cf0 0x7f6a800cfc10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6a800d4ad0 tx=0x7f6a6c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa4f7fa1e96417d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:28.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.823+0000 7f6a7ea00640 1 -- 172.21.15.26:0/918828615 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c037070 con 0x7f6a800d3cf0 2024-04-26T21:09:28.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.823+0000 7f6a7ea00640 1 -- 172.21.15.26:0/918828615 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a6c032440 con 0x7f6a800d3cf0 2024-04-26T21:09:28.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.823+0000 7f6a7ea00640 1 -- 172.21.15.26:0/918828615 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c032810 con 0x7f6a800d3cf0 2024-04-26T21:09:28.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.823+0000 7f6a85400640 1 -- 172.21.15.26:0/918828615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a800d3cf0 msgr2=0x7f6a800cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:28.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.823+0000 7f6a85400640 1 --2- 172.21.15.26:0/918828615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a800d3cf0 0x7f6a800cfc10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6a800d4ad0 tx=0x7f6a6c02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.824+0000 7f6a85400640 1 -- 172.21.15.26:0/918828615 shutdown_connections 2024-04-26T21:09:28.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.824+0000 7f6a85400640 1 --2- 172.21.15.26:0/918828615 >> v2:172.21.15.86:3300/0 conn(0x7f6a8015f4d0 0x7f6a801638f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.824+0000 7f6a85400640 1 --2- 172.21.15.26:0/918828615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a800d3cf0 0x7f6a800cfc10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.824+0000 7f6a85400640 1 -- 172.21.15.26:0/918828615 >> 172.21.15.26:0/918828615 conn(0x7f6a800d5000 msgr2=0x7f6a800d5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.824+0000 7f6a85400640 1 -- 172.21.15.26:0/918828615 shutdown_connections 2024-04-26T21:09:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.824+0000 7f6a85400640 1 -- 172.21.15.26:0/918828615 wait complete. 2024-04-26T21:09:28.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.825+0000 7f6a85400640 1 Processor -- start 2024-04-26T21:09:28.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.826+0000 7f6a85400640 1 -- start start 2024-04-26T21:09:28.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.826+0000 7f6a85400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a800d3cf0 0x7f6a801f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:28.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.826+0000 7f6a85400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a8015f0f0 0x7f6a800ce660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:28.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.826+0000 7f6a85400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a8015f4d0 0x7f6a800ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:28.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.827+0000 7f6a85400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6a800d0e30 con 0x7f6a8015f4d0 2024-04-26T21:09:28.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.827+0000 7f6a85400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6a800d0b50 con 0x7f6a800d3cf0 2024-04-26T21:09:28.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.827+0000 7f6a85400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6a800b5c30 con 0x7f6a8015f0f0 2024-04-26T21:09:28.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.827+0000 7f6a84a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a8015f4d0 0x7f6a800ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:28.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.827+0000 7f6a84a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a8015f4d0 0x7f6a800ceba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48644/0 (socket says 172.21.15.26:48644) 2024-04-26T21:09:28.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.827+0000 7f6a84a00640 1 -- 172.21.15.26:0/4034355641 learned_addr learned my addr 172.21.15.26:0/4034355641 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.827+0000 7f6a7f400640 1 --2- 172.21.15.26:0/4034355641 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a8015f0f0 0x7f6a800ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.827+0000 7f6a7fe00640 1 --2- 172.21.15.26:0/4034355641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a800d3cf0 0x7f6a801f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.827+0000 7f6a84a00640 1 -- 172.21.15.26:0/4034355641 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a8015f0f0 msgr2=0x7f6a800ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.827+0000 7f6a84a00640 1 --2- 172.21.15.26:0/4034355641 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a8015f0f0 0x7f6a800ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.827+0000 7f6a84a00640 1 -- 172.21.15.26:0/4034355641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a800d3cf0 msgr2=0x7f6a801f5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.827+0000 7f6a84a00640 1 --2- 172.21.15.26:0/4034355641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a800d3cf0 0x7f6a801f5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.827+0000 7f6a84a00640 1 -- 172.21.15.26:0/4034355641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a80160b50 con 0x7f6a8015f4d0 2024-04-26T21:09:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.827+0000 7f6a7fe00640 1 --2- 172.21.15.26:0/4034355641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a800d3cf0 0x7f6a801f5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:28.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.828+0000 7f6a84a00640 1 --2- 172.21.15.26:0/4034355641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a8015f4d0 0x7f6a800ceba0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6a78007e60 tx=0x7f6a7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:28.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.828+0000 7f6a77e00640 1 -- 172.21.15.26:0/4034355641 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78014420 con 0x7f6a8015f4d0 2024-04-26T21:09:28.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.828+0000 7f6a77e00640 1 -- 172.21.15.26:0/4034355641 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a78019070 con 0x7f6a8015f4d0 2024-04-26T21:09:28.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.828+0000 7f6a77e00640 1 -- 172.21.15.26:0/4034355641 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78014820 con 0x7f6a8015f4d0 2024-04-26T21:09:28.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.828+0000 7f6a85400640 1 -- 172.21.15.26:0/4034355641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a80160e30 con 0x7f6a8015f4d0 2024-04-26T21:09:28.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.828+0000 7f6a85400640 1 -- 172.21.15.26:0/4034355641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a801613f0 con 0x7f6a8015f4d0 2024-04-26T21:09:28.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.830+0000 7f6a85400640 1 -- 172.21.15.26:0/4034355641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a8006a700 con 0x7f6a8015f4d0 2024-04-26T21:09:28.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.831+0000 7f6a77e00640 1 -- 172.21.15.26:0/4034355641 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a780149c0 con 0x7f6a8015f4d0 2024-04-26T21:09:28.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.832+0000 7f6a77e00640 1 --2- 172.21.15.26:0/4034355641 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6a6006c8f0 0x7f6a6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:28.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.832+0000 7f6a77e00640 1 -- 172.21.15.26:0/4034355641 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f6a7808c3a0 con 0x7f6a8015f4d0 2024-04-26T21:09:28.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.832+0000 7f6a7fe00640 1 --2- 172.21.15.26:0/4034355641 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6a6006c8f0 0x7f6a6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:28.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.833+0000 7f6a7fe00640 1 --2- 172.21.15.26:0/4034355641 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6a6006c8f0 0x7f6a6006edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6a70000f60 tx=0x7f6a70009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:28.837+0000 7f6a77e00640 1 -- 172.21.15.26:0/4034355641 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6a7805a7e0 con 0x7f6a8015f4d0 2024-04-26T21:09:29.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.052+0000 7f6a85400640 1 -- 172.21.15.26:0/4034355641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6a8008d000 con 0x7f6a8015f4d0 2024-04-26T21:09:29.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.053+0000 7f6a77e00640 1 -- 172.21.15.26:0/4034355641 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f6a7805a0e0 con 0x7f6a8015f4d0 2024-04-26T21:09:29.053 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-04-26T21:09:29.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.055+0000 7f6a85400640 1 -- 172.21.15.26:0/4034355641 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6a6006c8f0 msgr2=0x7f6a6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:29.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.055+0000 7f6a85400640 1 --2- 172.21.15.26:0/4034355641 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6a6006c8f0 0x7f6a6006edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6a70000f60 tx=0x7f6a70009040 comp rx=0 tx=0).stop 2024-04-26T21:09:29.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.055+0000 7f6a85400640 1 -- 172.21.15.26:0/4034355641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a8015f4d0 msgr2=0x7f6a800ceba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:29.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.055+0000 7f6a85400640 1 --2- 172.21.15.26:0/4034355641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a8015f4d0 0x7f6a800ceba0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6a78007e60 tx=0x7f6a7800e4b0 comp rx=0 tx=0).stop 2024-04-26T21:09:29.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.056+0000 7f6a85400640 1 -- 172.21.15.26:0/4034355641 shutdown_connections 2024-04-26T21:09:29.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.056+0000 7f6a85400640 1 --2- 172.21.15.26:0/4034355641 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6a6006c8f0 0x7f6a6006edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:29.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.056+0000 7f6a85400640 1 --2- 172.21.15.26:0/4034355641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a8015f4d0 0x7f6a800ceba0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:29.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.056+0000 7f6a85400640 1 --2- 172.21.15.26:0/4034355641 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a8015f0f0 0x7f6a800ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:29.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.056+0000 7f6a85400640 1 --2- 172.21.15.26:0/4034355641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a800d3cf0 0x7f6a801f5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:29.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.056+0000 7f6a85400640 1 -- 172.21.15.26:0/4034355641 >> 172.21.15.26:0/4034355641 conn(0x7f6a800d5000 msgr2=0x7f6a800b99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:29.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.056+0000 7f6a85400640 1 -- 172.21.15.26:0/4034355641 shutdown_connections 2024-04-26T21:09:29.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.056+0000 7f6a85400640 1 -- 172.21.15.26:0/4034355641 wait complete. 2024-04-26T21:09:29.075 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-26T21:09:29.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:29.075 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-26T21:09:29.161 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-04-26T21:09:29.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:29.161 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-26T21:09:29.212 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:09:29.212 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:09:29.270 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:09:29.270 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:09:29.323 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:09:29.323 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:09:29.377 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:09:29.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.594+0000 7f76a6c00640 1 Processor -- start 2024-04-26T21:09:29.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.594+0000 7f76a6c00640 1 -- start start 2024-04-26T21:09:29.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.596+0000 7f76a6c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76a00d1400 0x7f76a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:29.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.596+0000 7f76a6c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76a0162e30 con 0x7f76a0148900 2024-04-26T21:09:29.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.596+0000 7f76a6c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76a01630a0 con 0x7f76a006e830 2024-04-26T21:09:29.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.596+0000 7f76a6c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f76a0163380 con 0x7f76a00d1400 2024-04-26T21:09:29.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.596+0000 7f769fe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f76a0148900 0x7f76a00d0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60794/0 (socket says 172.21.15.26:60794) 2024-04-26T21:09:29.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.597+0000 7f769fe00640 1 -- 172.21.15.26:0/1722193806 learned_addr learned my addr 172.21.15.26:0/1722193806 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:29.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.597+0000 7f769f400640 1 --2- 172.21.15.26:0/1722193806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76a00d1400 0x7f76a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:29.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.597+0000 7f769ea00640 1 -- 172.21.15.26:0/1722193806 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1252112107 0 0) 0x7f76a0162e30 con 0x7f76a0148900 2024-04-26T21:09:29.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.597+0000 7f769ea00640 1 -- 172.21.15.26:0/1722193806 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7684003520 con 0x7f76a0148900 2024-04-26T21:09:29.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.597+0000 7f769ea00640 1 -- 172.21.15.26:0/1722193806 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4264249485 0 0) 0x7f7684003520 con 0x7f76a0148900 2024-04-26T21:09:29.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.597+0000 7f769ea00640 1 -- 172.21.15.26:0/1722193806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76a00d1400 msgr2=0x7f76a006e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:29.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.597+0000 7f769ea00640 1 --2- 172.21.15.26:0/1722193806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76a00d1400 0x7f76a006e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.597+0000 7f769ea00640 1 -- 172.21.15.26:0/1722193806 >> v1:172.21.15.86:6789/0 conn(0x7f76a006e830 legacy=0x7f76a00cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.597+0000 7f769ea00640 1 -- 172.21.15.26:0/1722193806 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a0163660 con 0x7f76a0148900 2024-04-26T21:09:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.597+0000 7f769ea00640 1 -- 172.21.15.26:0/1722193806 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f7690001670 con 0x7f76a0148900 2024-04-26T21:09:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.597+0000 7f769ea00640 1 -- 172.21.15.26:0/1722193806 >> v1:172.21.15.26:6789/0 conn(0x7f76a0148900 legacy=0x7f76a00d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.598+0000 7f769ea00640 1 --2- 172.21.15.26:0/1722193806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76a00d1400 0x7f76840039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.598+0000 7f769ea00640 1 --2- 172.21.15.26:0/1722193806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7684004060 0x7f7684004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.598+0000 7f769ea00640 1 --2- 172.21.15.26:0/1722193806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7684004d40 0x7f76840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.598+0000 7f769ea00640 1 -- 172.21.15.26:0/1722193806 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f76a0163380 con 0x7f7684004d40 2024-04-26T21:09:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.598+0000 7f769ea00640 1 -- 172.21.15.26:0/1722193806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f768400bc00 con 0x7f7684004060 2024-04-26T21:09:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.598+0000 7f769ea00640 1 -- 172.21.15.26:0/1722193806 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f768400be00 con 0x7f76a00d1400 2024-04-26T21:09:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.598+0000 7f76a4c00640 1 --2- 172.21.15.26:0/1722193806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7684004d40 0x7f76840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.598+0000 7f769f400640 1 --2- 172.21.15.26:0/1722193806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7684004060 0x7f7684004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.598+0000 7f769fe00640 1 --2- 172.21.15.26:0/1722193806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76a00d1400 0x7f76840039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.598+0000 7f76a4c00640 1 -- 172.21.15.26:0/1722193806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76a00d1400 msgr2=0x7f76840039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:29.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.599+0000 7f76a4c00640 1 --2- 172.21.15.26:0/1722193806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76a00d1400 0x7f76840039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:29.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.599+0000 7f76a4c00640 1 -- 172.21.15.26:0/1722193806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7684004060 msgr2=0x7f7684004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:29.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.599+0000 7f76a4c00640 1 --2- 172.21.15.26:0/1722193806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7684004060 0x7f7684004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:29.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.599+0000 7f76a4c00640 1 -- 172.21.15.26:0/1722193806 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7684003520 con 0x7f7684004d40 2024-04-26T21:09:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.600+0000 7f76a4c00640 1 --2- 172.21.15.26:0/1722193806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7684004d40 0x7f76840051c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7698001e50 tx=0x7f7684005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd37368af2e1b475 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.600+0000 7f769ea00640 1 -- 172.21.15.26:0/1722193806 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7698001a70 con 0x7f7684004d40 2024-04-26T21:09:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.600+0000 7f769ea00640 1 -- 172.21.15.26:0/1722193806 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7698003970 con 0x7f7684004d40 2024-04-26T21:09:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.600+0000 7f769ea00640 1 -- 172.21.15.26:0/1722193806 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7698003d60 con 0x7f7684004d40 2024-04-26T21:09:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.600+0000 7f76a6c00640 1 -- 172.21.15.26:0/1722193806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7684004d40 msgr2=0x7f76840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.600+0000 7f76a6c00640 1 --2- 172.21.15.26:0/1722193806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7684004d40 0x7f76840051c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7698001e50 tx=0x7f7684005830 comp rx=0 tx=0).stop 2024-04-26T21:09:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.600+0000 7f76a4c00640 1 -- 172.21.15.26:0/1722193806 reap_dead start 2024-04-26T21:09:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.601+0000 7f76a6c00640 1 -- 172.21.15.26:0/1722193806 shutdown_connections 2024-04-26T21:09:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.601+0000 7f76a6c00640 1 -- 172.21.15.26:0/1722193806 >> 172.21.15.26:0/1722193806 conn(0x7f76a00cd840 msgr2=0x7f76a00cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.601+0000 7f76a6c00640 1 -- 172.21.15.26:0/1722193806 shutdown_connections 2024-04-26T21:09:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.601+0000 7f76a6c00640 1 -- 172.21.15.26:0/1722193806 wait complete. 2024-04-26T21:09:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.602+0000 7f76a6c00640 1 Processor -- start 2024-04-26T21:09:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.603+0000 7f76a6c00640 1 -- start start 2024-04-26T21:09:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.604+0000 7f76a6c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7684004d40 0x7f76a01471b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.604+0000 7f76a6c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76a013e2d0 0x7f76a013e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.604+0000 7f76a6c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76a01e8bb0 0x7f76a01e9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.604+0000 7f76a6c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f76a006ec90 con 0x7f76a01e8bb0 2024-04-26T21:09:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.604+0000 7f76a6c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f76a0148ef0 con 0x7f7684004d40 2024-04-26T21:09:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.604+0000 7f76a6c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f76a006ee10 con 0x7f76a013e2d0 2024-04-26T21:09:29.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.604+0000 7f76a4c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76a01e8bb0 0x7f76a01e9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:29.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.604+0000 7f76a4c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76a01e8bb0 0x7f76a01e9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48670/0 (socket says 172.21.15.26:48670) 2024-04-26T21:09:29.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.604+0000 7f76a4c00640 1 -- 172.21.15.26:0/233790807 learned_addr learned my addr 172.21.15.26:0/233790807 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:29.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.604+0000 7f769f400640 1 --2- 172.21.15.26:0/233790807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76a013e2d0 0x7f76a013e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:29.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.604+0000 7f769fe00640 1 --2- 172.21.15.26:0/233790807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7684004d40 0x7f76a01471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:29.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.605+0000 7f76a4c00640 1 -- 172.21.15.26:0/233790807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76a013e2d0 msgr2=0x7f76a013e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:29.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.605+0000 7f76a4c00640 1 --2- 172.21.15.26:0/233790807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76a013e2d0 0x7f76a013e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:29.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.605+0000 7f76a4c00640 1 -- 172.21.15.26:0/233790807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7684004d40 msgr2=0x7f76a01471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:29.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.605+0000 7f76a4c00640 1 --2- 172.21.15.26:0/233790807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7684004d40 0x7f76a01471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:29.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.605+0000 7f76a4c00640 1 -- 172.21.15.26:0/233790807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a0160450 con 0x7f76a01e8bb0 2024-04-26T21:09:29.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.605+0000 7f76a4c00640 1 --2- 172.21.15.26:0/233790807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76a01e8bb0 0x7f76a01e9010 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7698001f70 tx=0x7f76980014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:29.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.606+0000 7f7697e00640 1 -- 172.21.15.26:0/233790807 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7698037040 con 0x7f76a01e8bb0 2024-04-26T21:09:29.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.606+0000 7f7697e00640 1 -- 172.21.15.26:0/233790807 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7698003e50 con 0x7f76a01e8bb0 2024-04-26T21:09:29.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.606+0000 7f7697e00640 1 -- 172.21.15.26:0/233790807 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7698001a70 con 0x7f76a01e8bb0 2024-04-26T21:09:29.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.606+0000 7f76a6c00640 1 -- 172.21.15.26:0/233790807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a01f59c0 con 0x7f76a01e8bb0 2024-04-26T21:09:29.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.607+0000 7f76a6c00640 1 -- 172.21.15.26:0/233790807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a01f5f80 con 0x7f76a01e8bb0 2024-04-26T21:09:29.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.608+0000 7f76a6c00640 1 -- 172.21.15.26:0/233790807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a006a700 con 0x7f76a01e8bb0 2024-04-26T21:09:29.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.609+0000 7f7697e00640 1 -- 172.21.15.26:0/233790807 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7698042020 con 0x7f76a01e8bb0 2024-04-26T21:09:29.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.610+0000 7f7697e00640 1 --2- 172.21.15.26:0/233790807 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f768006c8f0 0x7f768006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:29.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.610+0000 7f7697e00640 1 -- 172.21.15.26:0/233790807 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f76980ab2e0 con 0x7f76a01e8bb0 2024-04-26T21:09:29.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.610+0000 7f769fe00640 1 --2- 172.21.15.26:0/233790807 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f768006c8f0 0x7f768006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:29.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.611+0000 7f769fe00640 1 --2- 172.21.15.26:0/233790807 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f768006c8f0 0x7f768006edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7690005ad0 tx=0x7f7690009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:29.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.615+0000 7f7697e00640 1 -- 172.21.15.26:0/233790807 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7698079670 con 0x7f76a01e8bb0 2024-04-26T21:09:29.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.820+0000 7f76a6c00640 1 -- 172.21.15.26:0/233790807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76a00d0f40 con 0x7f76a01e8bb0 2024-04-26T21:09:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.822+0000 7f7697e00640 1 -- 172.21.15.26:0/233790807 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16806 (secure 0 0 0) 0x7f7698078f70 con 0x7f76a01e8bb0 2024-04-26T21:09:29.823 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:29.823 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:09:24.561115+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:08:35.481169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:08:36.609498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:09:29.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.825+0000 7f76a6c00640 1 -- 172.21.15.26:0/233790807 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f768006c8f0 msgr2=0x7f768006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:29.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.825+0000 7f76a6c00640 1 --2- 172.21.15.26:0/233790807 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f768006c8f0 0x7f768006edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7690005ad0 tx=0x7f7690009040 comp rx=0 tx=0).stop 2024-04-26T21:09:29.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.825+0000 7f76a6c00640 1 -- 172.21.15.26:0/233790807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76a01e8bb0 msgr2=0x7f76a01e9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:29.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.825+0000 7f76a6c00640 1 --2- 172.21.15.26:0/233790807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76a01e8bb0 0x7f76a01e9010 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7698001f70 tx=0x7f76980014a0 comp rx=0 tx=0).stop 2024-04-26T21:09:29.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.825+0000 7f76a6c00640 1 -- 172.21.15.26:0/233790807 shutdown_connections 2024-04-26T21:09:29.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.825+0000 7f76a6c00640 1 --2- 172.21.15.26:0/233790807 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f768006c8f0 0x7f768006edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:29.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.825+0000 7f76a6c00640 1 --2- 172.21.15.26:0/233790807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76a01e8bb0 0x7f76a01e9010 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:29.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.825+0000 7f76a6c00640 1 --2- 172.21.15.26:0/233790807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76a013e2d0 0x7f76a013e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.825+0000 7f76a6c00640 1 --2- 172.21.15.26:0/233790807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7684004d40 0x7f76a01471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.825+0000 7f76a6c00640 1 -- 172.21.15.26:0/233790807 >> 172.21.15.26:0/233790807 conn(0x7f76a00cd840 msgr2=0x7f76a00af780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.825+0000 7f76a6c00640 1 -- 172.21.15.26:0/233790807 shutdown_connections 2024-04-26T21:09:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:29.825+0000 7f76a6c00640 1 -- 172.21.15.26:0/233790807 wait complete. 2024-04-26T21:09:29.846 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:08:31.322511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:08:35.481169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:08:36.609498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:09:29.847 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:09:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.053+0000 7f5e68600640 1 Processor -- start 2024-04-26T21:09:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.053+0000 7f5e68600640 1 -- start start 2024-04-26T21:09:30.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.055+0000 7f5e68600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5e600b5d30 0x7f5e600b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.055+0000 7f5e68600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e600b42c0 0x7f5e600b6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.055+0000 7f5e68600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e60146c00 con 0x7f5e600b6650 2024-04-26T21:09:30.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.055+0000 7f5e68600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f5e60073700 con 0x7f5e600b5d30 2024-04-26T21:09:30.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.055+0000 7f5e68600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5e6006e420 con 0x7f5e600b42c0 2024-04-26T21:09:30.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.055+0000 7f5e65c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5e600b5d30 0x7f5e600b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.055+0000 7f5e65c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5e600b5d30 0x7f5e600b3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48678/0 (socket says 172.21.15.26:48678) 2024-04-26T21:09:30.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.055+0000 7f5e65c00640 1 -- 172.21.15.26:0/3299241398 learned_addr learned my addr 172.21.15.26:0/3299241398 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:30.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.056+0000 7f5e65200640 1 --2- 172.21.15.26:0/3299241398 >> v2:172.21.15.86:3300/0 conn(0x7f5e600b42c0 0x7f5e600b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.056+0000 7f5e65c00640 1 -- 172.21.15.26:0/3299241398 >> v2:172.21.15.86:3300/0 conn(0x7f5e600b42c0 msgr2=0x7f5e600b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:30.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.056+0000 7f5e65c00640 1 --2- 172.21.15.26:0/3299241398 >> v2:172.21.15.86:3300/0 conn(0x7f5e600b42c0 0x7f5e600b6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.056+0000 7f5e65c00640 1 -- 172.21.15.26:0/3299241398 >> v1:172.21.15.86:6789/0 conn(0x7f5e600b6650 legacy=0x7f5e6006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:30.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.056+0000 7f5e65c00640 1 -- 172.21.15.26:0/3299241398 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e60072e60 con 0x7f5e600b5d30 2024-04-26T21:09:30.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.057+0000 7f5e65c00640 1 --2- 172.21.15.26:0/3299241398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e600b5d30 0x7f5e600b3d80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5e60074520 tx=0x7f5e5402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d88a187d45e380 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:30.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.058+0000 7f5e5fe00640 1 -- 172.21.15.26:0/3299241398 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e54037070 con 0x7f5e600b5d30 2024-04-26T21:09:30.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.058+0000 7f5e5fe00640 1 -- 172.21.15.26:0/3299241398 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e54032440 con 0x7f5e600b5d30 2024-04-26T21:09:30.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.058+0000 7f5e5fe00640 1 -- 172.21.15.26:0/3299241398 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e54032810 con 0x7f5e600b5d30 2024-04-26T21:09:30.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.059+0000 7f5e68600640 1 -- 172.21.15.26:0/3299241398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e600b5d30 msgr2=0x7f5e600b3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:30.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.059+0000 7f5e68600640 1 --2- 172.21.15.26:0/3299241398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e600b5d30 0x7f5e600b3d80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5e60074520 tx=0x7f5e5402a230 comp rx=0 tx=0).stop 2024-04-26T21:09:30.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.059+0000 7f5e68600640 1 -- 172.21.15.26:0/3299241398 shutdown_connections 2024-04-26T21:09:30.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.059+0000 7f5e68600640 1 --2- 172.21.15.26:0/3299241398 >> v2:172.21.15.86:3300/0 conn(0x7f5e600b42c0 0x7f5e600b6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.059+0000 7f5e68600640 1 --2- 172.21.15.26:0/3299241398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e600b5d30 0x7f5e600b3d80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.059+0000 7f5e68600640 1 -- 172.21.15.26:0/3299241398 >> 172.21.15.26:0/3299241398 conn(0x7f5e600b2a80 msgr2=0x7f5e600b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.059+0000 7f5e68600640 1 -- 172.21.15.26:0/3299241398 shutdown_connections 2024-04-26T21:09:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.059+0000 7f5e68600640 1 -- 172.21.15.26:0/3299241398 wait complete. 2024-04-26T21:09:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.061+0000 7f5e68600640 1 Processor -- start 2024-04-26T21:09:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.061+0000 7f5e68600640 1 -- start start 2024-04-26T21:09:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.062+0000 7f5e68600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e600b42c0 0x7f5e601f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.062+0000 7f5e68600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e600b5d30 0x7f5e601da390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.062+0000 7f5e68600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e600b6650 0x7f5e601e9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.062+0000 7f5e68600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5e60072a80 con 0x7f5e600b6650 2024-04-26T21:09:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.062+0000 7f5e68600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e60073420 con 0x7f5e600b5d30 2024-04-26T21:09:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.062+0000 7f5e68600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5e60072c00 con 0x7f5e600b42c0 2024-04-26T21:09:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.062+0000 7f5e66600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e600b6650 0x7f5e601e9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.062+0000 7f5e66600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e600b6650 0x7f5e601e9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48686/0 (socket says 172.21.15.26:48686) 2024-04-26T21:09:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.062+0000 7f5e66600640 1 -- 172.21.15.26:0/1725115023 learned_addr learned my addr 172.21.15.26:0/1725115023 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.063+0000 7f5e65c00640 1 --2- 172.21.15.26:0/1725115023 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e600b42c0 0x7f5e601f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.063+0000 7f5e65200640 1 --2- 172.21.15.26:0/1725115023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e600b5d30 0x7f5e601da390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.063+0000 7f5e66600640 1 -- 172.21.15.26:0/1725115023 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e600b42c0 msgr2=0x7f5e601f6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.063+0000 7f5e66600640 1 --2- 172.21.15.26:0/1725115023 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e600b42c0 0x7f5e601f6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.063+0000 7f5e66600640 1 -- 172.21.15.26:0/1725115023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e600b5d30 msgr2=0x7f5e601da390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.063+0000 7f5e66600640 1 --2- 172.21.15.26:0/1725115023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e600b5d30 0x7f5e601da390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.063+0000 7f5e66600640 1 -- 172.21.15.26:0/1725115023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e601e7100 con 0x7f5e600b6650 2024-04-26T21:09:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.063+0000 7f5e65200640 1 --2- 172.21.15.26:0/1725115023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e600b5d30 0x7f5e601da390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.063+0000 7f5e65c00640 1 --2- 172.21.15.26:0/1725115023 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e600b42c0 0x7f5e601f6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.063+0000 7f5e66600640 1 --2- 172.21.15.26:0/1725115023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e600b6650 0x7f5e601e9440 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800ba10 tx=0x7f5e5800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.063+0000 7f5e5d600640 1 -- 172.21.15.26:0/1725115023 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58019070 con 0x7f5e600b6650 2024-04-26T21:09:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.064+0000 7f5e5d600640 1 -- 172.21.15.26:0/1725115023 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e5800cb40 con 0x7f5e600b6650 2024-04-26T21:09:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.064+0000 7f5e5d600640 1 -- 172.21.15.26:0/1725115023 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58014440 con 0x7f5e600b6650 2024-04-26T21:09:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.064+0000 7f5e68600640 1 -- 172.21.15.26:0/1725115023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e601e9ee0 con 0x7f5e600b6650 2024-04-26T21:09:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.064+0000 7f5e68600640 1 -- 172.21.15.26:0/1725115023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e601e76a0 con 0x7f5e600b6650 2024-04-26T21:09:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.065+0000 7f5e68600640 1 -- 172.21.15.26:0/1725115023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e601e78a0 con 0x7f5e600b6650 2024-04-26T21:09:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.066+0000 7f5e5d600640 1 -- 172.21.15.26:0/1725115023 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e5801c070 con 0x7f5e600b6650 2024-04-26T21:09:30.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.067+0000 7f5e5d600640 1 --2- 172.21.15.26:0/1725115023 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5e4006c8f0 0x7f5e4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.067+0000 7f5e5d600640 1 -- 172.21.15.26:0/1725115023 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f5e5808b790 con 0x7f5e600b6650 2024-04-26T21:09:30.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.067+0000 7f5e65c00640 1 --2- 172.21.15.26:0/1725115023 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5e4006c8f0 0x7f5e4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.068+0000 7f5e65c00640 1 --2- 172.21.15.26:0/1725115023 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5e4006c8f0 0x7f5e4006edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5e54034000 tx=0x7f5e5402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:30.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.071+0000 7f5e5d600640 1 -- 172.21.15.26:0/1725115023 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5e601e78a0 con 0x7f5e600b6650 2024-04-26T21:09:30.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.281+0000 7f5e68600640 1 -- 172.21.15.26:0/1725115023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5e6006a700 con 0x7f5e600b6650 2024-04-26T21:09:30.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.282+0000 7f5e5d600640 1 -- 172.21.15.26:0/1725115023 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f5e58059b30 con 0x7f5e600b6650 2024-04-26T21:09:30.283 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-26T21:09:30.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.285+0000 7f5e68600640 1 -- 172.21.15.26:0/1725115023 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5e4006c8f0 msgr2=0x7f5e4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:30.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.285+0000 7f5e68600640 1 --2- 172.21.15.26:0/1725115023 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5e4006c8f0 0x7f5e4006edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5e54034000 tx=0x7f5e5402e000 comp rx=0 tx=0).stop 2024-04-26T21:09:30.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.285+0000 7f5e68600640 1 -- 172.21.15.26:0/1725115023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e600b6650 msgr2=0x7f5e601e9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:30.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.285+0000 7f5e68600640 1 --2- 172.21.15.26:0/1725115023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e600b6650 0x7f5e601e9440 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800ba10 tx=0x7f5e5800bed0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.285+0000 7f5e68600640 1 -- 172.21.15.26:0/1725115023 shutdown_connections 2024-04-26T21:09:30.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.285+0000 7f5e68600640 1 --2- 172.21.15.26:0/1725115023 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5e4006c8f0 0x7f5e4006edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.285+0000 7f5e68600640 1 --2- 172.21.15.26:0/1725115023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e600b6650 0x7f5e601e9440 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.285+0000 7f5e68600640 1 --2- 172.21.15.26:0/1725115023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e600b5d30 0x7f5e601da390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.285+0000 7f5e68600640 1 --2- 172.21.15.26:0/1725115023 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e600b42c0 0x7f5e601f6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.285+0000 7f5e68600640 1 -- 172.21.15.26:0/1725115023 >> 172.21.15.26:0/1725115023 conn(0x7f5e600b2a80 msgr2=0x7f5e601e6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:30.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.285+0000 7f5e68600640 1 -- 172.21.15.26:0/1725115023 shutdown_connections 2024-04-26T21:09:30.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.286+0000 7f5e68600640 1 -- 172.21.15.26:0/1725115023 wait complete. 2024-04-26T21:09:30.305 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:09:30.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.526+0000 7f4a06e00640 1 Processor -- start 2024-04-26T21:09:30.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.526+0000 7f4a06e00640 1 -- start start 2024-04-26T21:09:30.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.528+0000 7f4a06e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a000d22a0 0x7f4a0016a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.528+0000 7f4a06e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a000d2f50 con 0x7f4a000d0ff0 2024-04-26T21:09:30.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.528+0000 7f4a06e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a000d3230 con 0x7f4a0015f540 2024-04-26T21:09:30.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.528+0000 7f4a06e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4a000d5ed0 con 0x7f4a000d22a0 2024-04-26T21:09:30.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.528+0000 7f49ff400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4a000d0ff0 0x7f4a000d1b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60808/0 (socket says 172.21.15.26:60808) 2024-04-26T21:09:30.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.528+0000 7f49ff400640 1 -- 172.21.15.26:0/2531792002 learned_addr learned my addr 172.21.15.26:0/2531792002 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:30.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.528+0000 7f4a04e00640 1 --2- 172.21.15.26:0/2531792002 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a000d22a0 0x7f4a0016a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.529+0000 7f49fea00640 1 -- 172.21.15.26:0/2531792002 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3483366956 0 0) 0x7f4a000d2f50 con 0x7f4a000d0ff0 2024-04-26T21:09:30.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.529+0000 7f49fea00640 1 -- 172.21.15.26:0/2531792002 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49e4003520 con 0x7f4a000d0ff0 2024-04-26T21:09:30.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.529+0000 7f49fea00640 1 -- 172.21.15.26:0/2531792002 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2095229165 0 0) 0x7f49e4003520 con 0x7f4a000d0ff0 2024-04-26T21:09:30.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.529+0000 7f49fea00640 1 -- 172.21.15.26:0/2531792002 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a000d22a0 msgr2=0x7f4a0016a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:30.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.529+0000 7f49fea00640 1 --2- 172.21.15.26:0/2531792002 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a000d22a0 0x7f4a0016a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.529+0000 7f49fea00640 1 -- 172.21.15.26:0/2531792002 >> v1:172.21.15.86:6789/0 conn(0x7f4a0015f540 legacy=0x7f4a000d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:30.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.529+0000 7f49fea00640 1 -- 172.21.15.26:0/2531792002 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a000d3510 con 0x7f4a000d0ff0 2024-04-26T21:09:30.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.529+0000 7f49fea00640 1 -- 172.21.15.26:0/2531792002 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f49ec001670 con 0x7f4a000d0ff0 2024-04-26T21:09:30.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.529+0000 7f49fea00640 1 -- 172.21.15.26:0/2531792002 >> v1:172.21.15.26:6789/0 conn(0x7f4a000d0ff0 legacy=0x7f4a000d1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:30.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.529+0000 7f49fea00640 1 --2- 172.21.15.26:0/2531792002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49e40039e0 0x7f49e4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.529+0000 7f49fea00640 1 --2- 172.21.15.26:0/2531792002 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a000d22a0 0x7f49e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.529+0000 7f49fea00640 1 --2- 172.21.15.26:0/2531792002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e4004d40 0x7f49e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.529+0000 7f49fea00640 1 -- 172.21.15.26:0/2531792002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4a000d5ed0 con 0x7f49e40039e0 2024-04-26T21:09:30.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.529+0000 7f49fea00640 1 -- 172.21.15.26:0/2531792002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49e400bb80 con 0x7f49e4004d40 2024-04-26T21:09:30.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.529+0000 7f49fea00640 1 -- 172.21.15.26:0/2531792002 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f49e400bd80 con 0x7f4a000d22a0 2024-04-26T21:09:30.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.529+0000 7f49ffe00640 1 --2- 172.21.15.26:0/2531792002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49e40039e0 0x7f49e4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.530+0000 7f49ff400640 1 --2- 172.21.15.26:0/2531792002 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a000d22a0 0x7f49e4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.530+0000 7f4a04e00640 1 --2- 172.21.15.26:0/2531792002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e4004d40 0x7f49e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.530+0000 7f49ffe00640 1 -- 172.21.15.26:0/2531792002 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a000d22a0 msgr2=0x7f49e4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:30.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.530+0000 7f49ffe00640 1 --2- 172.21.15.26:0/2531792002 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a000d22a0 0x7f49e4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.530+0000 7f49ffe00640 1 -- 172.21.15.26:0/2531792002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e4004d40 msgr2=0x7f49e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:30.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.530+0000 7f49ffe00640 1 --2- 172.21.15.26:0/2531792002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e4004d40 0x7f49e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.530+0000 7f49ffe00640 1 -- 172.21.15.26:0/2531792002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e4003520 con 0x7f49e40039e0 2024-04-26T21:09:30.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.530+0000 7f4a04e00640 1 --2- 172.21.15.26:0/2531792002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49e4004d40 0x7f49e40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:30.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.531+0000 7f49ffe00640 1 --2- 172.21.15.26:0/2531792002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49e40039e0 0x7f49e4003e00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f49f0001750 tx=0x7f49e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6dbc575d1cff86 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:30.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.531+0000 7f49fea00640 1 -- 172.21.15.26:0/2531792002 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f0001370 con 0x7f49e40039e0 2024-04-26T21:09:30.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.531+0000 7f49fea00640 1 -- 172.21.15.26:0/2531792002 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49f00052d0 con 0x7f49e40039e0 2024-04-26T21:09:30.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.531+0000 7f49fea00640 1 -- 172.21.15.26:0/2531792002 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f002abe0 con 0x7f49e40039e0 2024-04-26T21:09:30.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.532+0000 7f4a06e00640 1 -- 172.21.15.26:0/2531792002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49e40039e0 msgr2=0x7f49e4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.532+0000 7f4a06e00640 1 --2- 172.21.15.26:0/2531792002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49e40039e0 0x7f49e4003e00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f49f0001750 tx=0x7f49e4005830 comp rx=0 tx=0).stop 2024-04-26T21:09:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.532+0000 7f4a04e00640 1 -- 172.21.15.26:0/2531792002 reap_dead start 2024-04-26T21:09:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.532+0000 7f4a06e00640 1 -- 172.21.15.26:0/2531792002 shutdown_connections 2024-04-26T21:09:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.532+0000 7f4a06e00640 1 -- 172.21.15.26:0/2531792002 >> 172.21.15.26:0/2531792002 conn(0x7f4a0009e520 msgr2=0x7f4a000d5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.532+0000 7f4a06e00640 1 -- 172.21.15.26:0/2531792002 shutdown_connections 2024-04-26T21:09:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.532+0000 7f4a06e00640 1 -- 172.21.15.26:0/2531792002 wait complete. 2024-04-26T21:09:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.534+0000 7f4a06e00640 1 Processor -- start 2024-04-26T21:09:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.534+0000 7f4a06e00640 1 -- start start 2024-04-26T21:09:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.535+0000 7f4a06e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49e40039e0 0x7f4a00158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.535+0000 7f4a06e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a000c8bf0 0x7f4a00148880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.535+0000 7f4a06e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a001f8c40 0x7f4a001f90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.535+0000 7f4a06e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0015f920 con 0x7f4a000c8bf0 2024-04-26T21:09:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.535+0000 7f4a06e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a000d3ad0 con 0x7f4a001f8c40 2024-04-26T21:09:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.535+0000 7f4a06e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4a000d3ff0 con 0x7f49e40039e0 2024-04-26T21:09:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.535+0000 7f49ff400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a000c8bf0 0x7f4a00148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.535+0000 7f49ff400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a000c8bf0 0x7f4a00148880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48700/0 (socket says 172.21.15.26:48700) 2024-04-26T21:09:30.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.535+0000 7f49ff400640 1 -- 172.21.15.26:0/3752353156 learned_addr learned my addr 172.21.15.26:0/3752353156 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:30.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.535+0000 7f49ffe00640 1 --2- 172.21.15.26:0/3752353156 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49e40039e0 0x7f4a00158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.536+0000 7f4a04e00640 1 --2- 172.21.15.26:0/3752353156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a001f8c40 0x7f4a001f90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.536+0000 7f49ff400640 1 -- 172.21.15.26:0/3752353156 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49e40039e0 msgr2=0x7f4a00158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:30.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.536+0000 7f49ff400640 1 --2- 172.21.15.26:0/3752353156 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49e40039e0 0x7f4a00158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.536+0000 7f49ff400640 1 -- 172.21.15.26:0/3752353156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a001f8c40 msgr2=0x7f4a001f90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:30.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.536+0000 7f49ff400640 1 --2- 172.21.15.26:0/3752353156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a001f8c40 0x7f4a001f90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.536+0000 7f49ff400640 1 -- 172.21.15.26:0/3752353156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a000b8fc0 con 0x7f4a000c8bf0 2024-04-26T21:09:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.536+0000 7f49ff400640 1 --2- 172.21.15.26:0/3752353156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a000c8bf0 0x7f4a00148880 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00d770 tx=0x7f49ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.536+0000 7f49f7e00640 1 -- 172.21.15.26:0/3752353156 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec019070 con 0x7f4a000c8bf0 2024-04-26T21:09:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.536+0000 7f49f7e00640 1 -- 172.21.15.26:0/3752353156 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49ec0074e0 con 0x7f4a000c8bf0 2024-04-26T21:09:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.536+0000 7f49f7e00640 1 -- 172.21.15.26:0/3752353156 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec004a00 con 0x7f4a000c8bf0 2024-04-26T21:09:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.536+0000 7f4a06e00640 1 -- 172.21.15.26:0/3752353156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a000b92a0 con 0x7f4a000c8bf0 2024-04-26T21:09:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.536+0000 7f4a06e00640 1 -- 172.21.15.26:0/3752353156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a000b9860 con 0x7f4a000c8bf0 2024-04-26T21:09:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.538+0000 7f4a06e00640 1 -- 172.21.15.26:0/3752353156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a000b9a60 con 0x7f4a000c8bf0 2024-04-26T21:09:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.539+0000 7f49f7e00640 1 -- 172.21.15.26:0/3752353156 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49ec004ba0 con 0x7f4a000c8bf0 2024-04-26T21:09:30.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.540+0000 7f49f7e00640 1 --2- 172.21.15.26:0/3752353156 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f49e006c920 0x7f49e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.540+0000 7f49f7e00640 1 -- 172.21.15.26:0/3752353156 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f49ec08b4e0 con 0x7f4a000c8bf0 2024-04-26T21:09:30.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.540+0000 7f49ffe00640 1 --2- 172.21.15.26:0/3752353156 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f49e006c920 0x7f49e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.541+0000 7f49ffe00640 1 --2- 172.21.15.26:0/3752353156 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f49e006c920 0x7f49e006ede0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f49f0032000 tx=0x7f49f002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:30.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.544+0000 7f49f7e00640 1 -- 172.21.15.26:0/3752353156 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4a000b9a60 con 0x7f4a000c8bf0 2024-04-26T21:09:30.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.748+0000 7f4a06e00640 1 -- 172.21.15.26:0/3752353156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4a0006a700 con 0x7f4a000c8bf0 2024-04-26T21:09:30.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.749+0000 7f49f7e00640 1 -- 172.21.15.26:0/3752353156 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f49ec00de20 con 0x7f4a000c8bf0 2024-04-26T21:09:30.750 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-26T21:09:30.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.752+0000 7f4a06e00640 1 -- 172.21.15.26:0/3752353156 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f49e006c920 msgr2=0x7f49e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:30.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.752+0000 7f4a06e00640 1 --2- 172.21.15.26:0/3752353156 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f49e006c920 0x7f49e006ede0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f49f0032000 tx=0x7f49f002e000 comp rx=0 tx=0).stop 2024-04-26T21:09:30.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.752+0000 7f4a06e00640 1 -- 172.21.15.26:0/3752353156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a000c8bf0 msgr2=0x7f4a00148880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:30.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.752+0000 7f4a06e00640 1 --2- 172.21.15.26:0/3752353156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a000c8bf0 0x7f4a00148880 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00d770 tx=0x7f49ec00dc30 comp rx=0 tx=0).stop 2024-04-26T21:09:30.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.752+0000 7f4a06e00640 1 -- 172.21.15.26:0/3752353156 shutdown_connections 2024-04-26T21:09:30.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.752+0000 7f4a06e00640 1 --2- 172.21.15.26:0/3752353156 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f49e006c920 0x7f49e006ede0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.752+0000 7f4a06e00640 1 --2- 172.21.15.26:0/3752353156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a001f8c40 0x7f4a001f90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.752+0000 7f4a06e00640 1 --2- 172.21.15.26:0/3752353156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a000c8bf0 0x7f4a00148880 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.752+0000 7f4a06e00640 1 --2- 172.21.15.26:0/3752353156 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49e40039e0 0x7f4a00158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.752+0000 7f4a06e00640 1 -- 172.21.15.26:0/3752353156 >> 172.21.15.26:0/3752353156 conn(0x7f4a0009e520 msgr2=0x7f4a0015faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:30.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.752+0000 7f4a06e00640 1 -- 172.21.15.26:0/3752353156 shutdown_connections 2024-04-26T21:09:30.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.752+0000 7f4a06e00640 1 -- 172.21.15.26:0/3752353156 wait complete. 2024-04-26T21:09:30.772 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:09:30.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.977+0000 7fb386c00640 1 Processor -- start 2024-04-26T21:09:30.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.977+0000 7fb386c00640 1 -- start start 2024-04-26T21:09:30.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.978+0000 7fb386c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3800b87b0 0x7fb380070f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.978+0000 7fb386c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb380072700 0x7fb380160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.978+0000 7fb386c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb3801610e0 0x7fb38015eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.978+0000 7fb386c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fb380161550 con 0x7fb3800b87b0 2024-04-26T21:09:30.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.978+0000 7fb386c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb380071470 con 0x7fb3801610e0 2024-04-26T21:09:30.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.978+0000 7fb386c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb3800cd970 con 0x7fb380072700 2024-04-26T21:09:30.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.979+0000 7fb37fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3800b87b0 0x7fb380070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.979+0000 7fb37fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3800b87b0 0x7fb380070f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48710/0 (socket says 172.21.15.26:48710) 2024-04-26T21:09:30.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.979+0000 7fb37fe00640 1 -- 172.21.15.26:0/2657792969 learned_addr learned my addr 172.21.15.26:0/2657792969 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:30.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.979+0000 7fb384c00640 1 --2- 172.21.15.26:0/2657792969 >> v2:172.21.15.86:3300/0 conn(0x7fb3801610e0 0x7fb38015eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.979+0000 7fb37f400640 1 --2- 172.21.15.26:0/2657792969 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb380072700 0x7fb380160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.979+0000 7fb37fe00640 1 -- 172.21.15.26:0/2657792969 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb380072700 msgr2=0x7fb380160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:30.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.979+0000 7fb37fe00640 1 --2- 172.21.15.26:0/2657792969 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb380072700 0x7fb380160ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.979+0000 7fb37fe00640 1 -- 172.21.15.26:0/2657792969 >> v2:172.21.15.86:3300/0 conn(0x7fb3801610e0 msgr2=0x7fb38015eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:30.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.979+0000 7fb37fe00640 1 --2- 172.21.15.26:0/2657792969 >> v2:172.21.15.86:3300/0 conn(0x7fb3801610e0 0x7fb38015eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.979+0000 7fb37fe00640 1 -- 172.21.15.26:0/2657792969 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb380071c20 con 0x7fb3800b87b0 2024-04-26T21:09:30.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.979+0000 7fb384c00640 1 --2- 172.21.15.26:0/2657792969 >> v2:172.21.15.86:3300/0 conn(0x7fb3801610e0 0x7fb38015eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:30.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.980+0000 7fb37fe00640 1 --2- 172.21.15.26:0/2657792969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3800b87b0 0x7fb380070f30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb3800cdb80 tx=0x7fb36802a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291e10aa45387c9d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:30.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.981+0000 7fb37ea00640 1 -- 172.21.15.26:0/2657792969 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368037070 con 0x7fb3800b87b0 2024-04-26T21:09:30.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.981+0000 7fb37ea00640 1 -- 172.21.15.26:0/2657792969 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb36802adf0 con 0x7fb3800b87b0 2024-04-26T21:09:30.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.981+0000 7fb37ea00640 1 -- 172.21.15.26:0/2657792969 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368032660 con 0x7fb3800b87b0 2024-04-26T21:09:30.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.981+0000 7fb386c00640 1 -- 172.21.15.26:0/2657792969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3800b87b0 msgr2=0x7fb380070f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:30.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.981+0000 7fb386c00640 1 --2- 172.21.15.26:0/2657792969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3800b87b0 0x7fb380070f30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb3800cdb80 tx=0x7fb36802a0a0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.981+0000 7fb386c00640 1 -- 172.21.15.26:0/2657792969 shutdown_connections 2024-04-26T21:09:30.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.981+0000 7fb386c00640 1 --2- 172.21.15.26:0/2657792969 >> v2:172.21.15.86:3300/0 conn(0x7fb3801610e0 0x7fb38015eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.981+0000 7fb386c00640 1 --2- 172.21.15.26:0/2657792969 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb380072700 0x7fb380160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.981+0000 7fb386c00640 1 --2- 172.21.15.26:0/2657792969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3800b87b0 0x7fb380070f30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.981+0000 7fb386c00640 1 -- 172.21.15.26:0/2657792969 >> 172.21.15.26:0/2657792969 conn(0x7fb3800b4b10 msgr2=0x7fb3800b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:30.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.981+0000 7fb386c00640 1 -- 172.21.15.26:0/2657792969 shutdown_connections 2024-04-26T21:09:30.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.981+0000 7fb386c00640 1 -- 172.21.15.26:0/2657792969 wait complete. 2024-04-26T21:09:30.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.983+0000 7fb386c00640 1 Processor -- start 2024-04-26T21:09:30.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.983+0000 7fb386c00640 1 -- start start 2024-04-26T21:09:30.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.984+0000 7fb386c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb380072700 0x7fb380155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.984+0000 7fb386c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3800b87b0 0x7fb3800b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.984+0000 7fb386c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3801610e0 0x7fb3800c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.984+0000 7fb386c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb380072b40 con 0x7fb3800b87b0 2024-04-26T21:09:30.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.984+0000 7fb386c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb3800721e0 con 0x7fb380072700 2024-04-26T21:09:30.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.984+0000 7fb386c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb3800cd5d0 con 0x7fb3801610e0 2024-04-26T21:09:30.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.985+0000 7fb37f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3800b87b0 0x7fb3800b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.985+0000 7fb37f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3800b87b0 0x7fb3800b5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48718/0 (socket says 172.21.15.26:48718) 2024-04-26T21:09:30.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.985+0000 7fb37f400640 1 -- 172.21.15.26:0/242420281 learned_addr learned my addr 172.21.15.26:0/242420281 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.985+0000 7fb37fe00640 1 --2- 172.21.15.26:0/242420281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb380072700 0x7fb380155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.985+0000 7fb384c00640 1 --2- 172.21.15.26:0/242420281 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3801610e0 0x7fb3800c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.985+0000 7fb37f400640 1 -- 172.21.15.26:0/242420281 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3801610e0 msgr2=0x7fb3800c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.985+0000 7fb37f400640 1 --2- 172.21.15.26:0/242420281 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3801610e0 0x7fb3800c6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.985+0000 7fb37f400640 1 -- 172.21.15.26:0/242420281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb380072700 msgr2=0x7fb380155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.985+0000 7fb37f400640 1 --2- 172.21.15.26:0/242420281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb380072700 0x7fb380155440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.985+0000 7fb37f400640 1 -- 172.21.15.26:0/242420281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3800b42c0 con 0x7fb3800b87b0 2024-04-26T21:09:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.985+0000 7fb37f400640 1 --2- 172.21.15.26:0/242420281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3800b87b0 0x7fb3800b5fb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb37000c820 tx=0x7fb37000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.986+0000 7fb377e00640 1 -- 172.21.15.26:0/242420281 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370015070 con 0x7fb3800b87b0 2024-04-26T21:09:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.986+0000 7fb377e00640 1 -- 172.21.15.26:0/242420281 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb370004240 con 0x7fb3800b87b0 2024-04-26T21:09:30.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.986+0000 7fb377e00640 1 -- 172.21.15.26:0/242420281 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37000ce10 con 0x7fb3800b87b0 2024-04-26T21:09:30.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.986+0000 7fb386c00640 1 -- 172.21.15.26:0/242420281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38006d350 con 0x7fb3800b87b0 2024-04-26T21:09:30.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.986+0000 7fb386c00640 1 -- 172.21.15.26:0/242420281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb38006d910 con 0x7fb3800b87b0 2024-04-26T21:09:30.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.987+0000 7fb386c00640 1 -- 172.21.15.26:0/242420281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb380050730 con 0x7fb3800b87b0 2024-04-26T21:09:30.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.988+0000 7fb377e00640 1 -- 172.21.15.26:0/242420281 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb370007830 con 0x7fb3800b87b0 2024-04-26T21:09:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.989+0000 7fb377e00640 1 --2- 172.21.15.26:0/242420281 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb35c06c8f0 0x7fb35c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.989+0000 7fb377e00640 1 -- 172.21.15.26:0/242420281 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb37008bac0 con 0x7fb3800b87b0 2024-04-26T21:09:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.990+0000 7fb37fe00640 1 --2- 172.21.15.26:0/242420281 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb35c06c8f0 0x7fb35c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:30.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.990+0000 7fb37fe00640 1 --2- 172.21.15.26:0/242420281 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb35c06c8f0 0x7fb35c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb368034000 tx=0x7fb36802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:30.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:30.994+0000 7fb377e00640 1 -- 172.21.15.26:0/242420281 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb3800b42c0 con 0x7fb3800b87b0 2024-04-26T21:09:31.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.199+0000 7fb386c00640 1 -- 172.21.15.26:0/242420281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb3800d29b0 con 0x7fb3800b87b0 2024-04-26T21:09:31.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.200+0000 7fb377e00640 1 -- 172.21.15.26:0/242420281 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fb370059e60 con 0x7fb3800b87b0 2024-04-26T21:09:31.201 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-26T21:09:31.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.203+0000 7fb386c00640 1 -- 172.21.15.26:0/242420281 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb35c06c8f0 msgr2=0x7fb35c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:31.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.203+0000 7fb386c00640 1 --2- 172.21.15.26:0/242420281 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb35c06c8f0 0x7fb35c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb368034000 tx=0x7fb36802e000 comp rx=0 tx=0).stop 2024-04-26T21:09:31.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.203+0000 7fb386c00640 1 -- 172.21.15.26:0/242420281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3800b87b0 msgr2=0x7fb3800b5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:31.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.203+0000 7fb386c00640 1 --2- 172.21.15.26:0/242420281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3800b87b0 0x7fb3800b5fb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb37000c820 tx=0x7fb37000cce0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.203+0000 7fb386c00640 1 -- 172.21.15.26:0/242420281 shutdown_connections 2024-04-26T21:09:31.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.203+0000 7fb386c00640 1 --2- 172.21.15.26:0/242420281 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb35c06c8f0 0x7fb35c06edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.203+0000 7fb386c00640 1 --2- 172.21.15.26:0/242420281 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3801610e0 0x7fb3800c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.203+0000 7fb386c00640 1 --2- 172.21.15.26:0/242420281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3800b87b0 0x7fb3800b5fb0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.203+0000 7fb386c00640 1 --2- 172.21.15.26:0/242420281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb380072700 0x7fb380155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.203+0000 7fb386c00640 1 -- 172.21.15.26:0/242420281 >> 172.21.15.26:0/242420281 conn(0x7fb3800b4b10 msgr2=0x7fb38015fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:31.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.203+0000 7fb386c00640 1 -- 172.21.15.26:0/242420281 shutdown_connections 2024-04-26T21:09:31.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.203+0000 7fb386c00640 1 -- 172.21.15.26:0/242420281 wait complete. 2024-04-26T21:09:31.223 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:09:31.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.430+0000 7fa1a7800640 1 Processor -- start 2024-04-26T21:09:31.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.431+0000 7fa1a7800640 1 -- start start 2024-04-26T21:09:31.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.432+0000 7fa1a7800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa1a006ec20 0x7fa1a00d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:31.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.432+0000 7fa1a7800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa1a00d1210 0x7fa1a0072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:31.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.432+0000 7fa1a7800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a00d1620 con 0x7fa1a0146990 2024-04-26T21:09:31.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.432+0000 7fa1a7800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a0146d70 con 0x7fa1a006ec20 2024-04-26T21:09:31.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.432+0000 7fa1a7800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a0073b70 con 0x7fa1a00d1210 2024-04-26T21:09:31.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.432+0000 7fa19fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa1a006ec20 0x7fa1a00d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:31.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.432+0000 7fa1a4e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa1a0146990 0x7fa1a00b3f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60812/0 (socket says 172.21.15.26:60812) 2024-04-26T21:09:31.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.432+0000 7fa1a4e00640 1 -- 172.21.15.26:0/4137019131 learned_addr learned my addr 172.21.15.26:0/4137019131 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:31.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.433+0000 7fa1a5800640 1 --2- 172.21.15.26:0/4137019131 >> v2:172.21.15.86:3300/0 conn(0x7fa1a00d1210 0x7fa1a0072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:31.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.433+0000 7fa19fe00640 1 -- 172.21.15.26:0/4137019131 >> v2:172.21.15.86:3300/0 conn(0x7fa1a00d1210 msgr2=0x7fa1a0072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:31.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.433+0000 7fa19fe00640 1 --2- 172.21.15.26:0/4137019131 >> v2:172.21.15.86:3300/0 conn(0x7fa1a00d1210 0x7fa1a0072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.433+0000 7fa19fe00640 1 -- 172.21.15.26:0/4137019131 >> v1:172.21.15.26:6789/0 conn(0x7fa1a0146990 legacy=0x7fa1a00b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:31.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.433+0000 7fa19fe00640 1 -- 172.21.15.26:0/4137019131 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a0073e50 con 0x7fa1a006ec20 2024-04-26T21:09:31.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.433+0000 7fa19f400640 1 -- 172.21.15.26:0/4137019131 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 778171682 0 0) 0x7fa190001490 con 0x7fa1a0146990 2024-04-26T21:09:31.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.433+0000 7fa1a5800640 1 --2- 172.21.15.26:0/4137019131 >> v2:172.21.15.86:3300/0 conn(0x7fa1a00d1210 0x7fa1a0072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:31.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.434+0000 7fa19fe00640 1 --2- 172.21.15.26:0/4137019131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1a006ec20 0x7fa1a00d0cd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0070610 tx=0x7fa18802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c40ae1aa3de320 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:31.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.434+0000 7fa19f400640 1 -- 172.21.15.26:0/4137019131 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188037070 con 0x7fa1a006ec20 2024-04-26T21:09:31.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.434+0000 7fa19f400640 1 -- 172.21.15.26:0/4137019131 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa188032440 con 0x7fa1a006ec20 2024-04-26T21:09:31.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.434+0000 7fa19f400640 1 -- 172.21.15.26:0/4137019131 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188032810 con 0x7fa1a006ec20 2024-04-26T21:09:31.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.435+0000 7fa1a7800640 1 -- 172.21.15.26:0/4137019131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1a006ec20 msgr2=0x7fa1a00d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:31.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.435+0000 7fa1a7800640 1 --2- 172.21.15.26:0/4137019131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1a006ec20 0x7fa1a00d0cd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0070610 tx=0x7fa18802a1a0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.435+0000 7fa1a7800640 1 -- 172.21.15.26:0/4137019131 shutdown_connections 2024-04-26T21:09:31.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.435+0000 7fa1a7800640 1 --2- 172.21.15.26:0/4137019131 >> v2:172.21.15.86:3300/0 conn(0x7fa1a00d1210 0x7fa1a0072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.435+0000 7fa1a7800640 1 --2- 172.21.15.26:0/4137019131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1a006ec20 0x7fa1a00d0cd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.435+0000 7fa1a7800640 1 -- 172.21.15.26:0/4137019131 >> 172.21.15.26:0/4137019131 conn(0x7fa1a00cd840 msgr2=0x7fa1a00cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:31.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.435+0000 7fa1a7800640 1 -- 172.21.15.26:0/4137019131 shutdown_connections 2024-04-26T21:09:31.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.435+0000 7fa1a7800640 1 -- 172.21.15.26:0/4137019131 wait complete. 2024-04-26T21:09:31.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.437+0000 7fa1a7800640 1 Processor -- start 2024-04-26T21:09:31.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.437+0000 7fa1a7800640 1 -- start start 2024-04-26T21:09:31.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.438+0000 7fa1a7800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1a006ec20 0x7fa1a0160270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:31.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.438+0000 7fa1a7800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa1a00d1210 0x7fa1a01f86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:31.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.438+0000 7fa1a7800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1a0146990 0x7fa1a01f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:31.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.438+0000 7fa1a7800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a006e700 con 0x7fa1a0146990 2024-04-26T21:09:31.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.438+0000 7fa1a7800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a006e420 con 0x7fa1a006ec20 2024-04-26T21:09:31.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.438+0000 7fa1a7800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a006f0b0 con 0x7fa1a00d1210 2024-04-26T21:09:31.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.438+0000 7fa1a5800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1a0146990 0x7fa1a01f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:31.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.438+0000 7fa1a5800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1a0146990 0x7fa1a01f8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48734/0 (socket says 172.21.15.26:48734) 2024-04-26T21:09:31.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.438+0000 7fa1a5800640 1 -- 172.21.15.26:0/2971795436 learned_addr learned my addr 172.21.15.26:0/2971795436 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:31.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.439+0000 7fa19fe00640 1 --2- 172.21.15.26:0/2971795436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa1a00d1210 0x7fa1a01f86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:31.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.439+0000 7fa1a4e00640 1 --2- 172.21.15.26:0/2971795436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1a006ec20 0x7fa1a0160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:31.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.439+0000 7fa1a5800640 1 -- 172.21.15.26:0/2971795436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa1a00d1210 msgr2=0x7fa1a01f86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:31.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.439+0000 7fa1a5800640 1 --2- 172.21.15.26:0/2971795436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa1a00d1210 0x7fa1a01f86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.439+0000 7fa1a5800640 1 -- 172.21.15.26:0/2971795436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1a006ec20 msgr2=0x7fa1a0160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:31.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.439+0000 7fa1a5800640 1 --2- 172.21.15.26:0/2971795436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1a006ec20 0x7fa1a0160270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.439+0000 7fa1a5800640 1 -- 172.21.15.26:0/2971795436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a01441d0 con 0x7fa1a0146990 2024-04-26T21:09:31.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.439+0000 7fa19fe00640 1 --2- 172.21.15.26:0/2971795436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa1a00d1210 0x7fa1a01f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:31.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.439+0000 7fa1a4e00640 1 --2- 172.21.15.26:0/2971795436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1a006ec20 0x7fa1a0160270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:31.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.439+0000 7fa1a5800640 1 --2- 172.21.15.26:0/2971795436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1a0146990 0x7fa1a01f8c30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa19800c7f0 tx=0x7fa19800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:31.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.439+0000 7fa19cc00640 1 -- 172.21.15.26:0/2971795436 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198015070 con 0x7fa1a0146990 2024-04-26T21:09:31.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.439+0000 7fa19cc00640 1 -- 172.21.15.26:0/2971795436 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa198007820 con 0x7fa1a0146990 2024-04-26T21:09:31.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.440+0000 7fa19cc00640 1 -- 172.21.15.26:0/2971795436 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198007c10 con 0x7fa1a0146990 2024-04-26T21:09:31.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.440+0000 7fa1a7800640 1 -- 172.21.15.26:0/2971795436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a01444b0 con 0x7fa1a0146990 2024-04-26T21:09:31.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.440+0000 7fa1a7800640 1 -- 172.21.15.26:0/2971795436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a01f6210 con 0x7fa1a0146990 2024-04-26T21:09:31.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.441+0000 7fa1a7800640 1 -- 172.21.15.26:0/2971795436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a0143f60 con 0x7fa1a0146990 2024-04-26T21:09:31.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.442+0000 7fa19cc00640 1 -- 172.21.15.26:0/2971795436 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa198018070 con 0x7fa1a0146990 2024-04-26T21:09:31.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.443+0000 7fa19cc00640 1 --2- 172.21.15.26:0/2971795436 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa17c06c8f0 0x7fa17c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:31.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.443+0000 7fa19cc00640 1 -- 172.21.15.26:0/2971795436 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fa19808bd70 con 0x7fa1a0146990 2024-04-26T21:09:31.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.443+0000 7fa1a4e00640 1 --2- 172.21.15.26:0/2971795436 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa17c06c8f0 0x7fa17c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:31.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.444+0000 7fa1a4e00640 1 --2- 172.21.15.26:0/2971795436 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa17c06c8f0 0x7fa17c06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa1900012e0 tx=0x7fa190009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:31.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.448+0000 7fa19cc00640 1 -- 172.21.15.26:0/2971795436 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa19805a110 con 0x7fa1a0146990 2024-04-26T21:09:31.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.668+0000 7fa1a7800640 1 -- 172.21.15.26:0/2971795436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa1a006fa90 con 0x7fa1a0146990 2024-04-26T21:09:31.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.669+0000 7fa19cc00640 1 -- 172.21.15.26:0/2971795436 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fa1a006fa90 con 0x7fa1a0146990 2024-04-26T21:09:31.669 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-04-26T21:09:31.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.671+0000 7fa1a7800640 1 -- 172.21.15.26:0/2971795436 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa17c06c8f0 msgr2=0x7fa17c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:31.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.671+0000 7fa1a7800640 1 --2- 172.21.15.26:0/2971795436 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa17c06c8f0 0x7fa17c06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa1900012e0 tx=0x7fa190009040 comp rx=0 tx=0).stop 2024-04-26T21:09:31.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.671+0000 7fa1a7800640 1 -- 172.21.15.26:0/2971795436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1a0146990 msgr2=0x7fa1a01f8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:31.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.671+0000 7fa1a7800640 1 --2- 172.21.15.26:0/2971795436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1a0146990 0x7fa1a01f8c30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa19800c7f0 tx=0x7fa19800ccb0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.672+0000 7fa1a7800640 1 -- 172.21.15.26:0/2971795436 shutdown_connections 2024-04-26T21:09:31.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.672+0000 7fa1a7800640 1 --2- 172.21.15.26:0/2971795436 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa17c06c8f0 0x7fa17c06edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.672+0000 7fa1a7800640 1 --2- 172.21.15.26:0/2971795436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1a0146990 0x7fa1a01f8c30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.672+0000 7fa1a7800640 1 --2- 172.21.15.26:0/2971795436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa1a00d1210 0x7fa1a01f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.672+0000 7fa1a7800640 1 --2- 172.21.15.26:0/2971795436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1a006ec20 0x7fa1a0160270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:31.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.672+0000 7fa1a7800640 1 -- 172.21.15.26:0/2971795436 >> 172.21.15.26:0/2971795436 conn(0x7fa1a00cd840 msgr2=0x7fa1a00b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:31.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.672+0000 7fa1a7800640 1 -- 172.21.15.26:0/2971795436 shutdown_connections 2024-04-26T21:09:31.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:31.672+0000 7fa1a7800640 1 -- 172.21.15.26:0/2971795436 wait complete. 2024-04-26T21:09:31.692 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-26T21:09:31.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:31.693 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-26T21:09:31.788 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-26T21:09:31.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:31.788 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-26T21:09:31.837 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T21:09:31.838 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:32.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.056+0000 7f398b200640 1 Processor -- start 2024-04-26T21:09:32.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.056+0000 7f398b200640 1 -- start start 2024-04-26T21:09:32.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.058+0000 7f398b200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f398c145c20 0x7f398c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:32.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.058+0000 7f398b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f398c0d1840 0x7f398c071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:32.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.058+0000 7f398b200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f398c0cf690 con 0x7f398c146450 2024-04-26T21:09:32.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.058+0000 7f398b200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f398c0ced10 con 0x7f398c145c20 2024-04-26T21:09:32.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.058+0000 7f398b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f398c0cf930 con 0x7f398c0d1840 2024-04-26T21:09:32.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.058+0000 7f3989400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f398c146450 0x7f398c0d1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60818/0 (socket says 172.21.15.26:60818) 2024-04-26T21:09:32.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.058+0000 7f3989400640 1 -- 172.21.15.26:0/1822714613 learned_addr learned my addr 172.21.15.26:0/1822714613 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:32.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.058+0000 7f3989e00640 1 --2- 172.21.15.26:0/1822714613 >> v2:172.21.15.26:3300/0 conn(0x7f398c145c20 0x7f398c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:32.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.059+0000 7f398a800640 1 --2- 172.21.15.26:0/1822714613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f398c0d1840 0x7f398c071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:32.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.059+0000 7f3988a00640 1 -- 172.21.15.26:0/1822714613 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2538573930 0 0) 0x7f398c0cf690 con 0x7f398c146450 2024-04-26T21:09:32.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.059+0000 7f3988a00640 1 -- 172.21.15.26:0/1822714613 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3970003520 con 0x7f398c146450 2024-04-26T21:09:32.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.059+0000 7f3989e00640 1 -- 172.21.15.26:0/1822714613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f398c0d1840 msgr2=0x7f398c071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:32.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.059+0000 7f3989e00640 1 --2- 172.21.15.26:0/1822714613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f398c0d1840 0x7f398c071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.059+0000 7f3989e00640 1 -- 172.21.15.26:0/1822714613 >> v1:172.21.15.26:6789/0 conn(0x7f398c146450 legacy=0x7f398c0d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:32.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.059+0000 7f3989e00640 1 -- 172.21.15.26:0/1822714613 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398c0cfc10 con 0x7f398c145c20 2024-04-26T21:09:32.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.060+0000 7f3989e00640 1 --2- 172.21.15.26:0/1822714613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f398c145c20 0x7f398c0d06f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f397c0097a0 tx=0x7f397c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90980ae59988f6b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:32.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.060+0000 7f3988a00640 1 -- 172.21.15.26:0/1822714613 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c037070 con 0x7f398c145c20 2024-04-26T21:09:32.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.061+0000 7f3988a00640 1 -- 172.21.15.26:0/1822714613 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f397c032440 con 0x7f398c145c20 2024-04-26T21:09:32.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.061+0000 7f3988a00640 1 -- 172.21.15.26:0/1822714613 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c0327f0 con 0x7f398c145c20 2024-04-26T21:09:32.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.061+0000 7f398b200640 1 -- 172.21.15.26:0/1822714613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f398c145c20 msgr2=0x7f398c0d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:32.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.061+0000 7f398b200640 1 --2- 172.21.15.26:0/1822714613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f398c145c20 0x7f398c0d06f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f397c0097a0 tx=0x7f397c02a180 comp rx=0 tx=0).stop 2024-04-26T21:09:32.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.061+0000 7f398b200640 1 -- 172.21.15.26:0/1822714613 shutdown_connections 2024-04-26T21:09:32.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.061+0000 7f398b200640 1 --2- 172.21.15.26:0/1822714613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f398c0d1840 0x7f398c071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.061+0000 7f398b200640 1 --2- 172.21.15.26:0/1822714613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f398c145c20 0x7f398c0d06f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.061+0000 7f398b200640 1 -- 172.21.15.26:0/1822714613 >> 172.21.15.26:0/1822714613 conn(0x7f398c06c690 msgr2=0x7f398c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:32.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.061+0000 7f398b200640 1 -- 172.21.15.26:0/1822714613 shutdown_connections 2024-04-26T21:09:32.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.061+0000 7f398b200640 1 -- 172.21.15.26:0/1822714613 wait complete. 2024-04-26T21:09:32.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.063+0000 7f398b200640 1 Processor -- start 2024-04-26T21:09:32.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.063+0000 7f398b200640 1 -- start start 2024-04-26T21:09:32.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.064+0000 7f398b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f398c0d1840 0x7f398c144880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:32.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.064+0000 7f398b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f398c145c20 0x7f398c142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.064+0000 7f398b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f398c146450 0x7f398c15e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.064+0000 7f398b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f398c146000 con 0x7f398c0d1840 2024-04-26T21:09:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.064+0000 7f398b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f398c0d01d0 con 0x7f398c145c20 2024-04-26T21:09:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.064+0000 7f398b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f398c15d2c0 con 0x7f398c146450 2024-04-26T21:09:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.065+0000 7f3989e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f398c0d1840 0x7f398c144880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.065+0000 7f3989e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f398c0d1840 0x7f398c144880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48754/0 (socket says 172.21.15.26:48754) 2024-04-26T21:09:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.065+0000 7f3989e00640 1 -- 172.21.15.26:0/4014521139 learned_addr learned my addr 172.21.15.26:0/4014521139 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.065+0000 7f398a800640 1 --2- 172.21.15.26:0/4014521139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f398c146450 0x7f398c15e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.065+0000 7f3989e00640 1 -- 172.21.15.26:0/4014521139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f398c146450 msgr2=0x7f398c15e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.065+0000 7f3989400640 1 --2- 172.21.15.26:0/4014521139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f398c145c20 0x7f398c142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.065+0000 7f3989e00640 1 --2- 172.21.15.26:0/4014521139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f398c146450 0x7f398c15e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.065+0000 7f3989e00640 1 -- 172.21.15.26:0/4014521139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f398c145c20 msgr2=0x7f398c142fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.065+0000 7f3989e00640 1 --2- 172.21.15.26:0/4014521139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f398c145c20 0x7f398c142fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.065+0000 7f3989e00640 1 -- 172.21.15.26:0/4014521139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f398c15d5a0 con 0x7f398c0d1840 2024-04-26T21:09:32.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.065+0000 7f3989e00640 1 --2- 172.21.15.26:0/4014521139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f398c0d1840 0x7f398c144880 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f397c035040 tx=0x7f397c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:32.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.066+0000 7f3982000640 1 -- 172.21.15.26:0/4014521139 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c041020 con 0x7f398c0d1840 2024-04-26T21:09:32.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.066+0000 7f3982000640 1 -- 172.21.15.26:0/4014521139 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f397c030700 con 0x7f398c0d1840 2024-04-26T21:09:32.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.066+0000 7f3982000640 1 -- 172.21.15.26:0/4014521139 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c037040 con 0x7f398c0d1840 2024-04-26T21:09:32.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.066+0000 7f398b200640 1 -- 172.21.15.26:0/4014521139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f398c15d880 con 0x7f398c0d1840 2024-04-26T21:09:32.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.066+0000 7f398b200640 1 -- 172.21.15.26:0/4014521139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f398c15de40 con 0x7f398c0d1840 2024-04-26T21:09:32.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.067+0000 7f398b200640 1 -- 172.21.15.26:0/4014521139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f398c15e040 con 0x7f398c0d1840 2024-04-26T21:09:32.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.068+0000 7f3982000640 1 -- 172.21.15.26:0/4014521139 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f397c044020 con 0x7f398c0d1840 2024-04-26T21:09:32.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.069+0000 7f3982000640 1 --2- 172.21.15.26:0/4014521139 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f396c06c940 0x7f396c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:32.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.069+0000 7f3982000640 1 -- 172.21.15.26:0/4014521139 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f397c0ac370 con 0x7f398c0d1840 2024-04-26T21:09:32.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.069+0000 7f3989400640 1 --2- 172.21.15.26:0/4014521139 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f396c06c940 0x7f396c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:32.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.070+0000 7f3989400640 1 --2- 172.21.15.26:0/4014521139 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f396c06c940 0x7f396c06ee00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f398c15ee90 tx=0x7f3978009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:32.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.074+0000 7f3982000640 1 -- 172.21.15.26:0/4014521139 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f398c15e040 con 0x7f398c0d1840 2024-04-26T21:09:32.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.316+0000 7f398b200640 1 -- 172.21.15.26:0/4014521139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f398c06a700 con 0x7f398c0d1840 2024-04-26T21:09:32.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.317+0000 7f3982000640 1 -- 172.21.15.26:0/4014521139 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7f397c04b030 con 0x7f398c0d1840 2024-04-26T21:09:32.318 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:32.318 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/2695371801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2695371801},{"type":"v1","addr":"172.21.15.26:6836","nonce":2695371801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3349732696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3349732696},{"type":"v1","addr":"172.21.15.26:6837","nonce":3349732696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/2288633870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2288633870},{"type":"v1","addr":"172.21.15.86:6833","nonce":2288633870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:08:37.622844+0000","modified":"2024-04-26T21:08:40.715140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/2125360615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2125360615},{"type":"v1","addr":"172.21.15.86:6835","nonce":2125360615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4261,"qdb_cluster":[4261]},"id":1}]} 2024-04-26T21:09:32.318 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2024-04-26T21:09:32.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.319+0000 7f398b200640 1 -- 172.21.15.26:0/4014521139 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f396c06c940 msgr2=0x7f396c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:32.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.319+0000 7f398b200640 1 --2- 172.21.15.26:0/4014521139 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f396c06c940 0x7f396c06ee00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f398c15ee90 tx=0x7f3978009040 comp rx=0 tx=0).stop 2024-04-26T21:09:32.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.319+0000 7f398b200640 1 -- 172.21.15.26:0/4014521139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f398c0d1840 msgr2=0x7f398c144880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:32.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.319+0000 7f398b200640 1 --2- 172.21.15.26:0/4014521139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f398c0d1840 0x7f398c144880 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f397c035040 tx=0x7f397c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.320+0000 7f398b200640 1 -- 172.21.15.26:0/4014521139 shutdown_connections 2024-04-26T21:09:32.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.320+0000 7f398b200640 1 --2- 172.21.15.26:0/4014521139 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f396c06c940 0x7f396c06ee00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.320+0000 7f398b200640 1 --2- 172.21.15.26:0/4014521139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f398c146450 0x7f398c15e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.320+0000 7f398b200640 1 --2- 172.21.15.26:0/4014521139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f398c145c20 0x7f398c142fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.320+0000 7f398b200640 1 --2- 172.21.15.26:0/4014521139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f398c0d1840 0x7f398c144880 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.320+0000 7f398b200640 1 -- 172.21.15.26:0/4014521139 >> 172.21.15.26:0/4014521139 conn(0x7f398c06c690 msgr2=0x7f398c0b1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:32.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.320+0000 7f398b200640 1 -- 172.21.15.26:0/4014521139 shutdown_connections 2024-04-26T21:09:32.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.320+0000 7f398b200640 1 -- 172.21.15.26:0/4014521139 wait complete. 2024-04-26T21:09:32.337 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:09:32.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.541+0000 7fce91400640 1 Processor -- start 2024-04-26T21:09:32.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.542+0000 7fce91400640 1 -- start start 2024-04-26T21:09:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.544+0000 7fce91400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fce8c073470 0x7fce8c073850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.544+0000 7fce91400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fce8c06f340 0x7fce8c15faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.544+0000 7fce91400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce8c06f7d0 con 0x7fce8c15ffe0 2024-04-26T21:09:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.544+0000 7fce91400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fce8c06fab0 con 0x7fce8c073470 2024-04-26T21:09:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.544+0000 7fce91400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fce8c06fd90 con 0x7fce8c06f340 2024-04-26T21:09:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.544+0000 7fce8be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fce8c073470 0x7fce8c073850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:32.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.544+0000 7fce8be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fce8c073470 0x7fce8c073850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48766/0 (socket says 172.21.15.26:48766) 2024-04-26T21:09:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.544+0000 7fce8be00640 1 -- 172.21.15.26:0/684521311 learned_addr learned my addr 172.21.15.26:0/684521311 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.544+0000 7fce83e00640 1 --2- 172.21.15.26:0/684521311 >> v2:172.21.15.86:3300/0 conn(0x7fce8c06f340 0x7fce8c15faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.545+0000 7fce8be00640 1 -- 172.21.15.26:0/684521311 >> v2:172.21.15.86:3300/0 conn(0x7fce8c06f340 msgr2=0x7fce8c15faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.545+0000 7fce8be00640 1 --2- 172.21.15.26:0/684521311 >> v2:172.21.15.86:3300/0 conn(0x7fce8c06f340 0x7fce8c15faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.545+0000 7fce8be00640 1 -- 172.21.15.26:0/684521311 >> v1:172.21.15.86:6789/0 conn(0x7fce8c15ffe0 legacy=0x7fce8c0d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.545+0000 7fce8be00640 1 -- 172.21.15.26:0/684521311 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8c070070 con 0x7fce8c073470 2024-04-26T21:09:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.546+0000 7fce8be00640 1 --2- 172.21.15.26:0/684521311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c073470 0x7fce8c073850 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fce8c0b4080 tx=0x7fce7c02c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4acda680df6817 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.546+0000 7fce8b400640 1 -- 172.21.15.26:0/684521311 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c039070 con 0x7fce8c073470 2024-04-26T21:09:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.546+0000 7fce8b400640 1 -- 172.21.15.26:0/684521311 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce7c034640 con 0x7fce8c073470 2024-04-26T21:09:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.546+0000 7fce8b400640 1 -- 172.21.15.26:0/684521311 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c034a10 con 0x7fce8c073470 2024-04-26T21:09:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.547+0000 7fce91400640 1 -- 172.21.15.26:0/684521311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c073470 msgr2=0x7fce8c073850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.547+0000 7fce91400640 1 --2- 172.21.15.26:0/684521311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c073470 0x7fce8c073850 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fce8c0b4080 tx=0x7fce7c02c3b0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.547+0000 7fce91400640 1 -- 172.21.15.26:0/684521311 shutdown_connections 2024-04-26T21:09:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.547+0000 7fce91400640 1 --2- 172.21.15.26:0/684521311 >> v2:172.21.15.86:3300/0 conn(0x7fce8c06f340 0x7fce8c15faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.547+0000 7fce91400640 1 --2- 172.21.15.26:0/684521311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c073470 0x7fce8c073850 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.547+0000 7fce91400640 1 -- 172.21.15.26:0/684521311 >> 172.21.15.26:0/684521311 conn(0x7fce8c06c690 msgr2=0x7fce8c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.547+0000 7fce91400640 1 -- 172.21.15.26:0/684521311 shutdown_connections 2024-04-26T21:09:32.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.547+0000 7fce91400640 1 -- 172.21.15.26:0/684521311 wait complete. 2024-04-26T21:09:32.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.548+0000 7fce91400640 1 Processor -- start 2024-04-26T21:09:32.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.549+0000 7fce91400640 1 -- start start 2024-04-26T21:09:32.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.550+0000 7fce91400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c073470 0x7fce8c148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:32.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.550+0000 7fce91400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce8c15ffe0 0x7fce8c157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:32.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.550+0000 7fce91400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce8c15e600 0x7fce8c0cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:32.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.550+0000 7fce91400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c073f10 con 0x7fce8c073470 2024-04-26T21:09:32.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.550+0000 7fce91400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fce8c073d90 con 0x7fce8c15ffe0 2024-04-26T21:09:32.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.550+0000 7fce91400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fce8c0d1900 con 0x7fce8c15e600 2024-04-26T21:09:32.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.550+0000 7fce8be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c073470 0x7fce8c148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:32.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.550+0000 7fce8be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c073470 0x7fce8c148790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48782/0 (socket says 172.21.15.26:48782) 2024-04-26T21:09:32.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.550+0000 7fce8be00640 1 -- 172.21.15.26:0/2130123869 learned_addr learned my addr 172.21.15.26:0/2130123869 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:32.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.550+0000 7fce83e00640 1 --2- 172.21.15.26:0/2130123869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce8c15ffe0 0x7fce8c157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:32.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.550+0000 7fce90a00640 1 --2- 172.21.15.26:0/2130123869 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce8c15e600 0x7fce8c0cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:32.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.550+0000 7fce8be00640 1 -- 172.21.15.26:0/2130123869 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce8c15e600 msgr2=0x7fce8c0cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:32.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.550+0000 7fce8be00640 1 --2- 172.21.15.26:0/2130123869 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce8c15e600 0x7fce8c0cf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.550+0000 7fce8be00640 1 -- 172.21.15.26:0/2130123869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce8c15ffe0 msgr2=0x7fce8c157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:32.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.550+0000 7fce8be00640 1 --2- 172.21.15.26:0/2130123869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce8c15ffe0 0x7fce8c157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.550+0000 7fce8be00640 1 -- 172.21.15.26:0/2130123869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8c148cd0 con 0x7fce8c073470 2024-04-26T21:09:32.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.550+0000 7fce83e00640 1 --2- 172.21.15.26:0/2130123869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce8c15ffe0 0x7fce8c157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:32.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.551+0000 7fce8be00640 1 --2- 172.21.15.26:0/2130123869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c073470 0x7fce8c148790 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fce7c000c00 tx=0x7fce7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:32.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.551+0000 7fce88c00640 1 -- 172.21.15.26:0/2130123869 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c039070 con 0x7fce8c073470 2024-04-26T21:09:32.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.551+0000 7fce88c00640 1 -- 172.21.15.26:0/2130123869 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce7c032980 con 0x7fce8c073470 2024-04-26T21:09:32.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.551+0000 7fce88c00640 1 -- 172.21.15.26:0/2130123869 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c032d50 con 0x7fce8c073470 2024-04-26T21:09:32.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.551+0000 7fce91400640 1 -- 172.21.15.26:0/2130123869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce8c157ef0 con 0x7fce8c073470 2024-04-26T21:09:32.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.551+0000 7fce91400640 1 -- 172.21.15.26:0/2130123869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce8c1584b0 con 0x7fce8c073470 2024-04-26T21:09:32.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.552+0000 7fce91400640 1 -- 172.21.15.26:0/2130123869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce8c06a700 con 0x7fce8c073470 2024-04-26T21:09:32.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.553+0000 7fce88c00640 1 -- 172.21.15.26:0/2130123869 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce7c03f440 con 0x7fce8c073470 2024-04-26T21:09:32.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.554+0000 7fce88c00640 1 --2- 172.21.15.26:0/2130123869 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fce6806c8f0 0x7fce6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:32.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.554+0000 7fce88c00640 1 -- 172.21.15.26:0/2130123869 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fce7c0ac800 con 0x7fce8c073470 2024-04-26T21:09:32.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.554+0000 7fce83e00640 1 --2- 172.21.15.26:0/2130123869 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fce6806c8f0 0x7fce6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:32.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.555+0000 7fce83e00640 1 --2- 172.21.15.26:0/2130123869 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fce6806c8f0 0x7fce6806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fce8c15dd70 tx=0x7fce74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:32.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.559+0000 7fce88c00640 1 -- 172.21.15.26:0/2130123869 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fce8c148cd0 con 0x7fce8c073470 2024-04-26T21:09:32.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.758+0000 7fce91400640 1 -- 172.21.15.26:0/2130123869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce8c15f4c0 con 0x7fce8c073470 2024-04-26T21:09:32.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.761+0000 7fce88c00640 1 -- 172.21.15.26:0/2130123869 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16806 (secure 0 0 0) 0x7fce7c07acb0 con 0x7fce8c073470 2024-04-26T21:09:32.763 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:32.763 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:09:24.561115+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:08:35.481169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:08:36.609498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:09:32.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.764+0000 7fce91400640 1 -- 172.21.15.26:0/2130123869 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fce6806c8f0 msgr2=0x7fce6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:32.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.764+0000 7fce91400640 1 --2- 172.21.15.26:0/2130123869 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fce6806c8f0 0x7fce6806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fce8c15dd70 tx=0x7fce74008040 comp rx=0 tx=0).stop 2024-04-26T21:09:32.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.764+0000 7fce91400640 1 -- 172.21.15.26:0/2130123869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c073470 msgr2=0x7fce8c148790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:32.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.764+0000 7fce91400640 1 --2- 172.21.15.26:0/2130123869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c073470 0x7fce8c148790 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fce7c000c00 tx=0x7fce7c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.764+0000 7fce91400640 1 -- 172.21.15.26:0/2130123869 shutdown_connections 2024-04-26T21:09:32.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.764+0000 7fce91400640 1 --2- 172.21.15.26:0/2130123869 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fce6806c8f0 0x7fce6806edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.764+0000 7fce91400640 1 --2- 172.21.15.26:0/2130123869 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce8c15e600 0x7fce8c0cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.764+0000 7fce91400640 1 --2- 172.21.15.26:0/2130123869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce8c15ffe0 0x7fce8c157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.764+0000 7fce91400640 1 --2- 172.21.15.26:0/2130123869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c073470 0x7fce8c148790 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.764+0000 7fce91400640 1 -- 172.21.15.26:0/2130123869 >> 172.21.15.26:0/2130123869 conn(0x7fce8c06c690 msgr2=0x7fce8c15a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:32.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.764+0000 7fce91400640 1 -- 172.21.15.26:0/2130123869 shutdown_connections 2024-04-26T21:09:32.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.764+0000 7fce91400640 1 -- 172.21.15.26:0/2130123869 wait complete. 2024-04-26T21:09:32.783 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:08:31.322511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:08:35.481169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:08:36.609498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:09:32.784 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:09:32.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.989+0000 7f4821c00640 1 Processor -- start 2024-04-26T21:09:32.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.989+0000 7f4821c00640 1 -- start start 2024-04-26T21:09:32.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.990+0000 7f4821c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f481c15bf30 0x7f481c072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:32.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.990+0000 7f4821c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f481c0b4a00 0x7f481c073240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:32.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.991+0000 7f4821c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f481c06e720 con 0x7f481c0727d0 2024-04-26T21:09:32.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.991+0000 7f4821c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f481c06e1f0 con 0x7f481c0b4a00 2024-04-26T21:09:32.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.991+0000 7f4821c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f481c0cf770 con 0x7f481c15bf30 2024-04-26T21:09:32.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.991+0000 7f481aa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f481c15bf30 0x7f481c072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:32.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.992+0000 7f481b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f481c0727d0 0x7f481c15fbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.26:53962/0 (socket says 172.21.15.26:53962) 2024-04-26T21:09:32.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.992+0000 7f4813e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f481c0b4a00 0x7f481c073240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:32.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.992+0000 7f481b400640 1 -- 172.21.15.26:0/519198824 learned_addr learned my addr 172.21.15.26:0/519198824 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:32.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.992+0000 7f481aa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f481c15bf30 0x7f481c072d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.26:51270/0 (socket says 172.21.15.26:51270) 2024-04-26T21:09:32.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.992+0000 7f481aa00640 1 -- 172.21.15.26:0/519198824 >> v2:172.21.15.86:3300/0 conn(0x7f481c0b4a00 msgr2=0x7f481c073240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:32.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.992+0000 7f481aa00640 1 --2- 172.21.15.26:0/519198824 >> v2:172.21.15.86:3300/0 conn(0x7f481c0b4a00 0x7f481c073240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.992+0000 7f481aa00640 1 -- 172.21.15.26:0/519198824 >> v1:172.21.15.86:6789/0 conn(0x7f481c0727d0 legacy=0x7f481c15fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:32.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.992+0000 7f481aa00640 1 -- 172.21.15.26:0/519198824 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481c06e440 con 0x7f481c15bf30 2024-04-26T21:09:32.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.992+0000 7f4813e00640 1 --2- 172.21.15.26:0/519198824 >> v2:172.21.15.86:3300/0 conn(0x7f481c0b4a00 0x7f481c073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:09:32.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.994+0000 7f481aa00640 1 --2- 172.21.15.26:0/519198824 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f481c15bf30 0x7f481c072d00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f480c0097a0 tx=0x7f480c02a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfbee19e9aa466d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:32.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.994+0000 7f481a000640 1 -- 172.21.15.26:0/519198824 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c037070 con 0x7f481c15bf30 2024-04-26T21:09:32.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.994+0000 7f481a000640 1 -- 172.21.15.26:0/519198824 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f480c02ae40 con 0x7f481c15bf30 2024-04-26T21:09:32.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.994+0000 7f481a000640 1 -- 172.21.15.26:0/519198824 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c032660 con 0x7f481c15bf30 2024-04-26T21:09:32.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.995+0000 7f4821c00640 1 -- 172.21.15.26:0/519198824 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f481c15bf30 msgr2=0x7f481c072d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:32.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.995+0000 7f4821c00640 1 --2- 172.21.15.26:0/519198824 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f481c15bf30 0x7f481c072d00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f480c0097a0 tx=0x7f480c02a0f0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.996+0000 7f4821c00640 1 -- 172.21.15.26:0/519198824 shutdown_connections 2024-04-26T21:09:32.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.996+0000 7f4821c00640 1 --2- 172.21.15.26:0/519198824 >> v2:172.21.15.86:3300/0 conn(0x7f481c0b4a00 0x7f481c073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.996+0000 7f4821c00640 1 --2- 172.21.15.26:0/519198824 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f481c15bf30 0x7f481c072d00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:32.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.996+0000 7f4821c00640 1 -- 172.21.15.26:0/519198824 >> 172.21.15.26:0/519198824 conn(0x7f481c0b2a80 msgr2=0x7f481c0b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:32.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.996+0000 7f4821c00640 1 -- 172.21.15.26:0/519198824 shutdown_connections 2024-04-26T21:09:32.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.996+0000 7f4821c00640 1 -- 172.21.15.26:0/519198824 wait complete. 2024-04-26T21:09:32.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.998+0000 7f4821c00640 1 Processor -- start 2024-04-26T21:09:32.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.998+0000 7f4821c00640 1 -- start start 2024-04-26T21:09:33.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.999+0000 7f4821c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f481c0727d0 0x7f481c140400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:33.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.999+0000 7f4821c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f481c0b4a00 0x7f481c151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:33.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.999+0000 7f4821c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f481c15bf30 0x7f481c158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:33.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.999+0000 7f4821c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f481c15c490 con 0x7f481c0b4a00 2024-04-26T21:09:33.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:32.999+0000 7f4821c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f481c15c310 con 0x7f481c0727d0 2024-04-26T21:09:33.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.000+0000 7f4821c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f481c073a80 con 0x7f481c15bf30 2024-04-26T21:09:33.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.000+0000 7f4813e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f481c0b4a00 0x7f481c151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:33.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.000+0000 7f4813e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f481c0b4a00 0x7f481c151eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48788/0 (socket says 172.21.15.26:48788) 2024-04-26T21:09:33.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.000+0000 7f4813e00640 1 -- 172.21.15.26:0/291519043 learned_addr learned my addr 172.21.15.26:0/291519043 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:33.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.000+0000 7f481b400640 1 --2- 172.21.15.26:0/291519043 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f481c15bf30 0x7f481c158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:33.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.000+0000 7f481aa00640 1 --2- 172.21.15.26:0/291519043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f481c0727d0 0x7f481c140400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:33.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.000+0000 7f4813e00640 1 -- 172.21.15.26:0/291519043 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f481c15bf30 msgr2=0x7f481c158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:33.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.000+0000 7f4813e00640 1 --2- 172.21.15.26:0/291519043 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f481c15bf30 0x7f481c158f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.000+0000 7f4813e00640 1 -- 172.21.15.26:0/291519043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f481c0727d0 msgr2=0x7f481c140400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:33.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.000+0000 7f4813e00640 1 --2- 172.21.15.26:0/291519043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f481c0727d0 0x7f481c140400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.000+0000 7f4813e00640 1 -- 172.21.15.26:0/291519043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481c1547d0 con 0x7f481c0b4a00 2024-04-26T21:09:33.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.001+0000 7f4813e00640 1 --2- 172.21.15.26:0/291519043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f481c0b4a00 0x7f481c151eb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f480400ebe0 tx=0x7f480400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:33.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.001+0000 7f4812a00640 1 -- 172.21.15.26:0/291519043 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804017070 con 0x7f481c0b4a00 2024-04-26T21:09:33.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.001+0000 7f4812a00640 1 -- 172.21.15.26:0/291519043 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f480400cc30 con 0x7f481c0b4a00 2024-04-26T21:09:33.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.001+0000 7f4812a00640 1 -- 172.21.15.26:0/291519043 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804013470 con 0x7f481c0b4a00 2024-04-26T21:09:33.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.001+0000 7f4821c00640 1 -- 172.21.15.26:0/291519043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481c154ab0 con 0x7f481c0b4a00 2024-04-26T21:09:33.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.001+0000 7f4821c00640 1 -- 172.21.15.26:0/291519043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f481c155070 con 0x7f481c0b4a00 2024-04-26T21:09:33.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.002+0000 7f4821c00640 1 -- 172.21.15.26:0/291519043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f481c06a700 con 0x7f481c0b4a00 2024-04-26T21:09:33.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.003+0000 7f4812a00640 1 -- 172.21.15.26:0/291519043 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f480401a070 con 0x7f481c0b4a00 2024-04-26T21:09:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.004+0000 7f4812a00640 1 --2- 172.21.15.26:0/291519043 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f47f806c8f0 0x7f47f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.004+0000 7f4812a00640 1 -- 172.21.15.26:0/291519043 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f480408bf60 con 0x7f481c0b4a00 2024-04-26T21:09:33.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.005+0000 7f481aa00640 1 --2- 172.21.15.26:0/291519043 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f47f806c8f0 0x7f47f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:33.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.006+0000 7f481aa00640 1 --2- 172.21.15.26:0/291519043 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f47f806c8f0 0x7f47f806edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f480c034000 tx=0x7f480c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:33.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.009+0000 7f4812a00640 1 -- 172.21.15.26:0/291519043 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f480405a270 con 0x7f481c0b4a00 2024-04-26T21:09:33.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.207+0000 7f4821c00640 1 -- 172.21.15.26:0/291519043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f481c06fe30 con 0x7f481c0b4a00 2024-04-26T21:09:33.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.208+0000 7f4812a00640 1 -- 172.21.15.26:0/291519043 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f4804059b70 con 0x7f481c0b4a00 2024-04-26T21:09:33.209 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-26T21:09:33.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.211+0000 7f4821c00640 1 -- 172.21.15.26:0/291519043 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f47f806c8f0 msgr2=0x7f47f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:33.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.211+0000 7f4821c00640 1 --2- 172.21.15.26:0/291519043 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f47f806c8f0 0x7f47f806edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f480c034000 tx=0x7f480c02e000 comp rx=0 tx=0).stop 2024-04-26T21:09:33.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.211+0000 7f4821c00640 1 -- 172.21.15.26:0/291519043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f481c0b4a00 msgr2=0x7f481c151eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:33.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.211+0000 7f4821c00640 1 --2- 172.21.15.26:0/291519043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f481c0b4a00 0x7f481c151eb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f480400ebe0 tx=0x7f480400c520 comp rx=0 tx=0).stop 2024-04-26T21:09:33.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.211+0000 7f4821c00640 1 -- 172.21.15.26:0/291519043 shutdown_connections 2024-04-26T21:09:33.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.211+0000 7f4821c00640 1 --2- 172.21.15.26:0/291519043 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f47f806c8f0 0x7f47f806edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.211+0000 7f4821c00640 1 --2- 172.21.15.26:0/291519043 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f481c15bf30 0x7f481c158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.211+0000 7f4821c00640 1 --2- 172.21.15.26:0/291519043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f481c0b4a00 0x7f481c151eb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.211+0000 7f4821c00640 1 --2- 172.21.15.26:0/291519043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f481c0727d0 0x7f481c140400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.211+0000 7f4821c00640 1 -- 172.21.15.26:0/291519043 >> 172.21.15.26:0/291519043 conn(0x7f481c0b2a80 msgr2=0x7f481c0d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:33.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.211+0000 7f4821c00640 1 -- 172.21.15.26:0/291519043 shutdown_connections 2024-04-26T21:09:33.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.211+0000 7f4821c00640 1 -- 172.21.15.26:0/291519043 wait complete. 2024-04-26T21:09:33.230 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:09:33.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.434+0000 7f6e10a00640 1 Processor -- start 2024-04-26T21:09:33.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.435+0000 7f6e10a00640 1 -- start start 2024-04-26T21:09:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.436+0000 7f6e10a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e0c162e60 0x7f6e0c1632e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.436+0000 7f6e10a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6e0c0b8640 0x7f6e0c0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.436+0000 7f6e10a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e0c0d3870 con 0x7f6e0c15f290 2024-04-26T21:09:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.436+0000 7f6e10a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f6e0c0d3b50 con 0x7f6e0c162e60 2024-04-26T21:09:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.436+0000 7f6e10a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6e0c0d3e30 con 0x7f6e0c0b8640 2024-04-26T21:09:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.437+0000 7f6e0aa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e0c162e60 0x7f6e0c1632e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:33.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.437+0000 7f6e0aa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e0c162e60 0x7f6e0c1632e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48798/0 (socket says 172.21.15.26:48798) 2024-04-26T21:09:33.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.437+0000 7f6e0aa00640 1 -- 172.21.15.26:0/332593404 learned_addr learned my addr 172.21.15.26:0/332593404 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:33.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.437+0000 7f6e0be00640 1 --2- 172.21.15.26:0/332593404 >> v2:172.21.15.86:3300/0 conn(0x7f6e0c0b8640 0x7f6e0c0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:33.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.437+0000 7f6e0aa00640 1 -- 172.21.15.26:0/332593404 >> v2:172.21.15.86:3300/0 conn(0x7f6e0c0b8640 msgr2=0x7f6e0c0b8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:33.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.437+0000 7f6e0aa00640 1 --2- 172.21.15.26:0/332593404 >> v2:172.21.15.86:3300/0 conn(0x7f6e0c0b8640 0x7f6e0c0b8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.437+0000 7f6e0aa00640 1 -- 172.21.15.26:0/332593404 >> v1:172.21.15.86:6789/0 conn(0x7f6e0c15f290 legacy=0x7f6e0c0d5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:33.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.437+0000 7f6e0aa00640 1 -- 172.21.15.26:0/332593404 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0c0b7e40 con 0x7f6e0c162e60 2024-04-26T21:09:33.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.438+0000 7f6e0aa00640 1 --2- 172.21.15.26:0/332593404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0c162e60 0x7f6e0c1632e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c08f560 tx=0x7f6df402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49abc4c0315671aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:33.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.439+0000 7f6e0a000640 1 -- 172.21.15.26:0/332593404 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4037070 con 0x7f6e0c162e60 2024-04-26T21:09:33.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.439+0000 7f6e0a000640 1 -- 172.21.15.26:0/332593404 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6df4032440 con 0x7f6e0c162e60 2024-04-26T21:09:33.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.439+0000 7f6e0a000640 1 -- 172.21.15.26:0/332593404 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4032810 con 0x7f6e0c162e60 2024-04-26T21:09:33.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.439+0000 7f6e10a00640 1 -- 172.21.15.26:0/332593404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0c162e60 msgr2=0x7f6e0c1632e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:33.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.439+0000 7f6e10a00640 1 --2- 172.21.15.26:0/332593404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0c162e60 0x7f6e0c1632e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c08f560 tx=0x7f6df402a230 comp rx=0 tx=0).stop 2024-04-26T21:09:33.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.440+0000 7f6e10a00640 1 -- 172.21.15.26:0/332593404 shutdown_connections 2024-04-26T21:09:33.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.440+0000 7f6e10a00640 1 --2- 172.21.15.26:0/332593404 >> v2:172.21.15.86:3300/0 conn(0x7f6e0c0b8640 0x7f6e0c0b8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.440+0000 7f6e10a00640 1 --2- 172.21.15.26:0/332593404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0c162e60 0x7f6e0c1632e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.440+0000 7f6e10a00640 1 -- 172.21.15.26:0/332593404 >> 172.21.15.26:0/332593404 conn(0x7f6e0c09e380 msgr2=0x7f6e0c0d4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:33.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.440+0000 7f6e10a00640 1 -- 172.21.15.26:0/332593404 shutdown_connections 2024-04-26T21:09:33.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.440+0000 7f6e10a00640 1 -- 172.21.15.26:0/332593404 wait complete. 2024-04-26T21:09:33.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.442+0000 7f6e10a00640 1 Processor -- start 2024-04-26T21:09:33.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.442+0000 7f6e10a00640 1 -- start start 2024-04-26T21:09:33.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.443+0000 7f6e10a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0c0b8640 0x7f6e0c0ba370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:33.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.443+0000 7f6e10a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e0c15f290 0x7f6e0c0ba8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:33.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.443+0000 7f6e10a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e0c162e60 0x7f6e0c0c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:33.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.443+0000 7f6e10a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c0d3540 con 0x7f6e0c0b8640 2024-04-26T21:09:33.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.443+0000 7f6e10a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c0d5e10 con 0x7f6e0c15f290 2024-04-26T21:09:33.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.443+0000 7f6e10a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6e0c15f670 con 0x7f6e0c162e60 2024-04-26T21:09:33.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.444+0000 7f6e0b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0c0b8640 0x7f6e0c0ba370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:33.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.444+0000 7f6e0b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0c0b8640 0x7f6e0c0ba370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48806/0 (socket says 172.21.15.26:48806) 2024-04-26T21:09:33.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.444+0000 7f6e0b400640 1 -- 172.21.15.26:0/4080488651 learned_addr learned my addr 172.21.15.26:0/4080488651 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:33.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.444+0000 7f6e0b400640 1 -- 172.21.15.26:0/4080488651 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e0c162e60 msgr2=0x7f6e0c0c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:33.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.444+0000 7f6e0b400640 1 --2- 172.21.15.26:0/4080488651 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e0c162e60 0x7f6e0c0c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.444+0000 7f6e0b400640 1 -- 172.21.15.26:0/4080488651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e0c15f290 msgr2=0x7f6e0c0ba8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:33.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.444+0000 7f6e0aa00640 1 --2- 172.21.15.26:0/4080488651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e0c15f290 0x7f6e0c0ba8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:33.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.444+0000 7f6e0b400640 1 --2- 172.21.15.26:0/4080488651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e0c15f290 0x7f6e0c0ba8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.444+0000 7f6e0b400640 1 -- 172.21.15.26:0/4080488651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0c0ca9b0 con 0x7f6e0c0b8640 2024-04-26T21:09:33.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.444+0000 7f6e0b400640 1 --2- 172.21.15.26:0/4080488651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0c0b8640 0x7f6e0c0ba370 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00c430 tx=0x7f6dfc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:33.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.445+0000 7f6e03400640 1 -- 172.21.15.26:0/4080488651 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc011790 con 0x7f6e0c0b8640 2024-04-26T21:09:33.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.445+0000 7f6e03400640 1 -- 172.21.15.26:0/4080488651 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dfc017070 con 0x7f6e0c0b8640 2024-04-26T21:09:33.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.445+0000 7f6e03400640 1 -- 172.21.15.26:0/4080488651 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc011bb0 con 0x7f6e0c0b8640 2024-04-26T21:09:33.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.445+0000 7f6e10a00640 1 -- 172.21.15.26:0/4080488651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e0c0c9910 con 0x7f6e0c0b8640 2024-04-26T21:09:33.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.445+0000 7f6e10a00640 1 -- 172.21.15.26:0/4080488651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e0c0c9ed0 con 0x7f6e0c0b8640 2024-04-26T21:09:33.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.446+0000 7f6e10a00640 1 -- 172.21.15.26:0/4080488651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e0c06a700 con 0x7f6e0c0b8640 2024-04-26T21:09:33.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.447+0000 7f6e03400640 1 -- 172.21.15.26:0/4080488651 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6dfc01c460 con 0x7f6e0c0b8640 2024-04-26T21:09:33.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.448+0000 7f6e03400640 1 --2- 172.21.15.26:0/4080488651 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6de806c920 0x7f6de806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:33.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.448+0000 7f6e03400640 1 -- 172.21.15.26:0/4080488651 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f6dfc08baa0 con 0x7f6e0c0b8640 2024-04-26T21:09:33.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.448+0000 7f6e0aa00640 1 --2- 172.21.15.26:0/4080488651 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6de806c920 0x7f6de806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:33.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.449+0000 7f6e0aa00640 1 --2- 172.21.15.26:0/4080488651 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6de806c920 0x7f6de806ede0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6df4034000 tx=0x7f6df402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:33.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.452+0000 7f6e03400640 1 -- 172.21.15.26:0/4080488651 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6dfc059db0 con 0x7f6e0c0b8640 2024-04-26T21:09:33.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.661+0000 7f6e10a00640 1 -- 172.21.15.26:0/4080488651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6e0c0d23e0 con 0x7f6e0c0b8640 2024-04-26T21:09:33.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.662+0000 7f6e03400640 1 -- 172.21.15.26:0/4080488651 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f6dfc0596b0 con 0x7f6e0c0b8640 2024-04-26T21:09:33.663 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-26T21:09:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.665+0000 7f6e10a00640 1 -- 172.21.15.26:0/4080488651 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6de806c920 msgr2=0x7f6de806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.665+0000 7f6e10a00640 1 --2- 172.21.15.26:0/4080488651 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6de806c920 0x7f6de806ede0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6df4034000 tx=0x7f6df402e000 comp rx=0 tx=0).stop 2024-04-26T21:09:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.665+0000 7f6e10a00640 1 -- 172.21.15.26:0/4080488651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0c0b8640 msgr2=0x7f6e0c0ba370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.665+0000 7f6e10a00640 1 --2- 172.21.15.26:0/4080488651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0c0b8640 0x7f6e0c0ba370 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00c430 tx=0x7f6dfc00c8f0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.665+0000 7f6e10a00640 1 -- 172.21.15.26:0/4080488651 shutdown_connections 2024-04-26T21:09:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.665+0000 7f6e10a00640 1 --2- 172.21.15.26:0/4080488651 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6de806c920 0x7f6de806ede0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.665+0000 7f6e10a00640 1 --2- 172.21.15.26:0/4080488651 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e0c162e60 0x7f6e0c0c8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.665+0000 7f6e10a00640 1 --2- 172.21.15.26:0/4080488651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e0c15f290 0x7f6e0c0ba8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.665+0000 7f6e10a00640 1 --2- 172.21.15.26:0/4080488651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0c0b8640 0x7f6e0c0ba370 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.665+0000 7f6e10a00640 1 -- 172.21.15.26:0/4080488651 >> 172.21.15.26:0/4080488651 conn(0x7f6e0c09e380 msgr2=0x7f6e0c15cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.665+0000 7f6e10a00640 1 -- 172.21.15.26:0/4080488651 shutdown_connections 2024-04-26T21:09:33.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.665+0000 7f6e10a00640 1 -- 172.21.15.26:0/4080488651 wait complete. 2024-04-26T21:09:33.685 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:09:33.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.900+0000 7f0edb200640 1 Processor -- start 2024-04-26T21:09:33.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.900+0000 7f0edb200640 1 -- start start 2024-04-26T21:09:33.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.902+0000 7f0edb200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0ed4164d10 0x7f0ed4167100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:33.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.902+0000 7f0edb200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ed40b41e0 con 0x7f0ed40cfce0 2024-04-26T21:09:33.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.902+0000 7f0edb200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ed415faa0 con 0x7f0ed40b3d80 2024-04-26T21:09:33.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.902+0000 7f0edb200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f0ed415fd80 con 0x7f0ed4164d10 2024-04-26T21:09:33.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.902+0000 7f0ed9e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0ed40cfce0 0x7f0ed40d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60824/0 (socket says 172.21.15.26:60824) 2024-04-26T21:09:33.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.902+0000 7f0ed9e00640 1 -- 172.21.15.26:0/2205015046 learned_addr learned my addr 172.21.15.26:0/2205015046 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:33.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.902+0000 7f0eda800640 1 --2- 172.21.15.26:0/2205015046 >> v2:172.21.15.26:3300/0 conn(0x7f0ed4164d10 0x7f0ed4167100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:33.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.903+0000 7f0eda800640 1 -- 172.21.15.26:0/2205015046 >> v1:172.21.15.86:6789/0 conn(0x7f0ed40b3d80 legacy=0x7f0ed415cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:33.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.903+0000 7f0eda800640 1 -- 172.21.15.26:0/2205015046 >> v1:172.21.15.26:6789/0 conn(0x7f0ed40cfce0 legacy=0x7f0ed40d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:33.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.903+0000 7f0eda800640 1 -- 172.21.15.26:0/2205015046 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed4160060 con 0x7f0ed4164d10 2024-04-26T21:09:33.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.904+0000 7f0eda800640 1 --2- 172.21.15.26:0/2205015046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ed4164d10 0x7f0ed4167100 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009d90 tx=0x7f0ed002c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1308eb871141bb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:33.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.904+0000 7f0ed8a00640 1 -- 172.21.15.26:0/2205015046 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0032770 con 0x7f0ed4164d10 2024-04-26T21:09:33.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.904+0000 7f0ed8a00640 1 -- 172.21.15.26:0/2205015046 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ed0038070 con 0x7f0ed4164d10 2024-04-26T21:09:33.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.904+0000 7f0ed8a00640 1 -- 172.21.15.26:0/2205015046 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0032b30 con 0x7f0ed4164d10 2024-04-26T21:09:33.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.905+0000 7f0edb200640 1 -- 172.21.15.26:0/2205015046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ed4164d10 msgr2=0x7f0ed4167100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:33.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.905+0000 7f0edb200640 1 --2- 172.21.15.26:0/2205015046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ed4164d10 0x7f0ed4167100 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009d90 tx=0x7f0ed002c8b0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.905+0000 7f0edb200640 1 -- 172.21.15.26:0/2205015046 shutdown_connections 2024-04-26T21:09:33.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.905+0000 7f0edb200640 1 --2- 172.21.15.26:0/2205015046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ed4164d10 0x7f0ed4167100 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.905+0000 7f0edb200640 1 -- 172.21.15.26:0/2205015046 >> 172.21.15.26:0/2205015046 conn(0x7f0ed40cd7d0 msgr2=0x7f0ed40cdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:33.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.905+0000 7f0edb200640 1 -- 172.21.15.26:0/2205015046 shutdown_connections 2024-04-26T21:09:33.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.905+0000 7f0edb200640 1 -- 172.21.15.26:0/2205015046 wait complete. 2024-04-26T21:09:33.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.907+0000 7f0edb200640 1 Processor -- start 2024-04-26T21:09:33.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.907+0000 7f0edb200640 1 -- start start 2024-04-26T21:09:33.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.908+0000 7f0edb200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ed40b3d80 0x7f0ed414a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:33.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.908+0000 7f0edb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ed40cfce0 0x7f0ed41519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:33.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.908+0000 7f0edb200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ed4164d10 0x7f0ed4151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:33.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.908+0000 7f0edb200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed4160860 con 0x7f0ed40b3d80 2024-04-26T21:09:33.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.908+0000 7f0edb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed40b4480 con 0x7f0ed40cfce0 2024-04-26T21:09:33.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.908+0000 7f0edb200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed415d250 con 0x7f0ed4164d10 2024-04-26T21:09:33.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.908+0000 7f0ed9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ed40b3d80 0x7f0ed414a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:33.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.908+0000 7f0ed9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ed40b3d80 0x7f0ed414a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48826/0 (socket says 172.21.15.26:48826) 2024-04-26T21:09:33.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.908+0000 7f0ed9e00640 1 -- 172.21.15.26:0/3745413524 learned_addr learned my addr 172.21.15.26:0/3745413524 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:33.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.909+0000 7f0ed9e00640 1 -- 172.21.15.26:0/3745413524 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ed4164d10 msgr2=0x7f0ed4151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:33.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.909+0000 7f0ed9e00640 1 --2- 172.21.15.26:0/3745413524 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ed4164d10 0x7f0ed4151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.909+0000 7f0ed9e00640 1 -- 172.21.15.26:0/3745413524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ed40cfce0 msgr2=0x7f0ed41519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:33.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.909+0000 7f0ed9e00640 1 --2- 172.21.15.26:0/3745413524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ed40cfce0 0x7f0ed41519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:33.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.909+0000 7f0ed9e00640 1 -- 172.21.15.26:0/3745413524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed41557f0 con 0x7f0ed40b3d80 2024-04-26T21:09:33.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.909+0000 7f0ed9e00640 1 --2- 172.21.15.26:0/3745413524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ed40b3d80 0x7f0ed414a8e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800c430 tx=0x7f0ec800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:33.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.909+0000 7f0ece000640 1 -- 172.21.15.26:0/3745413524 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8017070 con 0x7f0ed40b3d80 2024-04-26T21:09:33.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.909+0000 7f0ece000640 1 -- 172.21.15.26:0/3745413524 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ec80074e0 con 0x7f0ed40b3d80 2024-04-26T21:09:33.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.909+0000 7f0ece000640 1 -- 172.21.15.26:0/3745413524 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec80059a0 con 0x7f0ed40b3d80 2024-04-26T21:09:33.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.910+0000 7f0edb200640 1 -- 172.21.15.26:0/3745413524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed40b1810 con 0x7f0ed40b3d80 2024-04-26T21:09:33.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.910+0000 7f0edb200640 1 -- 172.21.15.26:0/3745413524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed414c690 con 0x7f0ed40b3d80 2024-04-26T21:09:33.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.911+0000 7f0edb200640 1 -- 172.21.15.26:0/3745413524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ed40506c0 con 0x7f0ed40b3d80 2024-04-26T21:09:33.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.912+0000 7f0ece000640 1 -- 172.21.15.26:0/3745413524 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ec801a070 con 0x7f0ed40b3d80 2024-04-26T21:09:33.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.913+0000 7f0ece000640 1 --2- 172.21.15.26:0/3745413524 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0eb806c8f0 0x7f0eb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:33.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.913+0000 7f0ece000640 1 -- 172.21.15.26:0/3745413524 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f0ec808af10 con 0x7f0ed40b3d80 2024-04-26T21:09:33.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.913+0000 7f0ed9400640 1 --2- 172.21.15.26:0/3745413524 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0eb806c8f0 0x7f0eb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:33.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.914+0000 7f0ed9400640 1 --2- 172.21.15.26:0/3745413524 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0eb806c8f0 0x7f0eb806edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4164120 tx=0x7f0ec40079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:33.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:33.917+0000 7f0ece000640 1 -- 172.21.15.26:0/3745413524 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0ec8058bd0 con 0x7f0ed40b3d80 2024-04-26T21:09:34.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.121+0000 7f0edb200640 1 -- 172.21.15.26:0/3745413524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0ed414ff40 con 0x7f0ed40b3d80 2024-04-26T21:09:34.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.121+0000 7f0ece000640 1 -- 172.21.15.26:0/3745413524 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f0ec805ced0 con 0x7f0ed40b3d80 2024-04-26T21:09:34.122 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-26T21:09:34.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.124+0000 7f0edb200640 1 -- 172.21.15.26:0/3745413524 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0eb806c8f0 msgr2=0x7f0eb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:34.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.124+0000 7f0edb200640 1 --2- 172.21.15.26:0/3745413524 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0eb806c8f0 0x7f0eb806edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4164120 tx=0x7f0ec40079b0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.124+0000 7f0edb200640 1 -- 172.21.15.26:0/3745413524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ed40b3d80 msgr2=0x7f0ed414a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.124+0000 7f0edb200640 1 --2- 172.21.15.26:0/3745413524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ed40b3d80 0x7f0ed414a8e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800c430 tx=0x7f0ec800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.124+0000 7f0edb200640 1 -- 172.21.15.26:0/3745413524 shutdown_connections 2024-04-26T21:09:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.124+0000 7f0edb200640 1 --2- 172.21.15.26:0/3745413524 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0eb806c8f0 0x7f0eb806edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.124+0000 7f0edb200640 1 --2- 172.21.15.26:0/3745413524 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ed4164d10 0x7f0ed4151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.124+0000 7f0edb200640 1 --2- 172.21.15.26:0/3745413524 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ed40cfce0 0x7f0ed41519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.124+0000 7f0edb200640 1 --2- 172.21.15.26:0/3745413524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ed40b3d80 0x7f0ed414a8e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.124+0000 7f0edb200640 1 -- 172.21.15.26:0/3745413524 >> 172.21.15.26:0/3745413524 conn(0x7f0ed40cd7d0 msgr2=0x7f0ed4153110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:34.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.124+0000 7f0edb200640 1 -- 172.21.15.26:0/3745413524 shutdown_connections 2024-04-26T21:09:34.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.124+0000 7f0edb200640 1 -- 172.21.15.26:0/3745413524 wait complete. 2024-04-26T21:09:34.145 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.359+0000 7fd19a600640 1 Processor -- start 2024-04-26T21:09:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.359+0000 7fd19a600640 1 -- start start 2024-04-26T21:09:34.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.361+0000 7fd19a600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd1940d06f0 0x7fd1940d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:34.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.361+0000 7fd19a600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1940cf690 con 0x7fd1940d1670 2024-04-26T21:09:34.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.361+0000 7fd19a600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1940cf930 con 0x7fd194145c20 2024-04-26T21:09:34.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.361+0000 7fd19a600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd1940ced10 con 0x7fd1940d06f0 2024-04-26T21:09:34.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.361+0000 7fd199c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd1940d1670 0x7fd194071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60838/0 (socket says 172.21.15.26:60838) 2024-04-26T21:09:34.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.361+0000 7fd199c00640 1 -- 172.21.15.26:0/3695042306 learned_addr learned my addr 172.21.15.26:0/3695042306 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:34.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.362+0000 7fd193e00640 1 --2- 172.21.15.26:0/3695042306 >> v2:172.21.15.86:3300/0 conn(0x7fd1940d06f0 0x7fd1940d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:34.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.362+0000 7fd193400640 1 -- 172.21.15.26:0/3695042306 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 952142518 0 0) 0x7fd1940cf690 con 0x7fd1940d1670 2024-04-26T21:09:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.362+0000 7fd193400640 1 -- 172.21.15.26:0/3695042306 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd17c003520 con 0x7fd1940d1670 2024-04-26T21:09:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.362+0000 7fd193400640 1 -- 172.21.15.26:0/3695042306 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 357123530 0 0) 0x7fd17c003520 con 0x7fd1940d1670 2024-04-26T21:09:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.362+0000 7fd193400640 1 -- 172.21.15.26:0/3695042306 >> v2:172.21.15.86:3300/0 conn(0x7fd1940d06f0 msgr2=0x7fd1940d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.362+0000 7fd193400640 1 --2- 172.21.15.26:0/3695042306 >> v2:172.21.15.86:3300/0 conn(0x7fd1940d06f0 0x7fd1940d1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.362+0000 7fd193400640 1 -- 172.21.15.26:0/3695042306 >> v1:172.21.15.86:6789/0 conn(0x7fd194145c20 legacy=0x7fd194146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.362+0000 7fd193400640 1 -- 172.21.15.26:0/3695042306 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1940cfc10 con 0x7fd1940d1670 2024-04-26T21:09:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.362+0000 7fd193400640 1 -- 172.21.15.26:0/3695042306 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7fd18c001a10 con 0x7fd1940d1670 2024-04-26T21:09:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.362+0000 7fd193400640 1 -- 172.21.15.26:0/3695042306 >> v1:172.21.15.26:6789/0 conn(0x7fd1940d1670 legacy=0x7fd194071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.362+0000 7fd193400640 1 --2- 172.21.15.26:0/3695042306 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd17c0039b0 0x7fd17c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.362+0000 7fd193400640 1 --2- 172.21.15.26:0/3695042306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd17c004800 0x7fd17c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.362+0000 7fd193400640 1 --2- 172.21.15.26:0/3695042306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd17c0051c0 0x7fd17c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.362+0000 7fd193400640 1 -- 172.21.15.26:0/3695042306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c00c030 con 0x7fd17c004800 2024-04-26T21:09:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.362+0000 7fd193400640 1 -- 172.21.15.26:0/3695042306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c00c1e0 con 0x7fd17c0051c0 2024-04-26T21:09:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.362+0000 7fd193400640 1 -- 172.21.15.26:0/3695042306 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd17c00c3e0 con 0x7fd17c0039b0 2024-04-26T21:09:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.363+0000 7fd193e00640 1 --2- 172.21.15.26:0/3695042306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd17c004800 0x7fd17c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.363+0000 7fd199200640 1 --2- 172.21.15.26:0/3695042306 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd17c0039b0 0x7fd17c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.363+0000 7fd193e00640 1 -- 172.21.15.26:0/3695042306 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd17c0039b0 msgr2=0x7fd17c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.363+0000 7fd193e00640 1 --2- 172.21.15.26:0/3695042306 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd17c0039b0 0x7fd17c003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.363+0000 7fd193e00640 1 -- 172.21.15.26:0/3695042306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd17c0051c0 msgr2=0x7fd17c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.363+0000 7fd193e00640 1 --2- 172.21.15.26:0/3695042306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd17c0051c0 0x7fd17c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.363+0000 7fd193e00640 1 -- 172.21.15.26:0/3695042306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17c003520 con 0x7fd17c004800 2024-04-26T21:09:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.363+0000 7fd199c00640 1 -- 172.21.15.26:0/3695042306 reap_dead start 2024-04-26T21:09:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.364+0000 7fd193e00640 1 --2- 172.21.15.26:0/3695042306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd17c004800 0x7fd17c004c80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd180009ac0 tx=0x7fd17c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5b0529e5f7c0b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.364+0000 7fd193400640 1 -- 172.21.15.26:0/3695042306 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180036070 con 0x7fd17c004800 2024-04-26T21:09:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.364+0000 7fd193400640 1 -- 172.21.15.26:0/3695042306 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd180031400 con 0x7fd17c004800 2024-04-26T21:09:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.364+0000 7fd193400640 1 -- 172.21.15.26:0/3695042306 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180009690 con 0x7fd17c004800 2024-04-26T21:09:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.365+0000 7fd19a600640 1 -- 172.21.15.26:0/3695042306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd17c004800 msgr2=0x7fd17c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.365+0000 7fd19a600640 1 --2- 172.21.15.26:0/3695042306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd17c004800 0x7fd17c004c80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd180009ac0 tx=0x7fd17c005830 comp rx=0 tx=0).stop 2024-04-26T21:09:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.365+0000 7fd19a600640 1 -- 172.21.15.26:0/3695042306 shutdown_connections 2024-04-26T21:09:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.365+0000 7fd19a600640 1 --2- 172.21.15.26:0/3695042306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd17c004800 0x7fd17c004c80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.365+0000 7fd19a600640 1 -- 172.21.15.26:0/3695042306 >> 172.21.15.26:0/3695042306 conn(0x7fd19406c690 msgr2=0x7fd19406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.365+0000 7fd19a600640 1 -- 172.21.15.26:0/3695042306 shutdown_connections 2024-04-26T21:09:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.365+0000 7fd19a600640 1 -- 172.21.15.26:0/3695042306 wait complete. 2024-04-26T21:09:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.367+0000 7fd19a600640 1 Processor -- start 2024-04-26T21:09:34.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.367+0000 7fd19a600640 1 -- start start 2024-04-26T21:09:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.368+0000 7fd19a600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd17c004800 0x7fd1941f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.368+0000 7fd19a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd19415d740 0x7fd1941f2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.368+0000 7fd19a600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd19415fdc0 0x7fd194143b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.368+0000 7fd19a600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd1940d1aa0 con 0x7fd17c004800 2024-04-26T21:09:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.368+0000 7fd19a600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1940d01d0 con 0x7fd19415d740 2024-04-26T21:09:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.368+0000 7fd19a600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd194146000 con 0x7fd19415fdc0 2024-04-26T21:09:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.368+0000 7fd199200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd17c004800 0x7fd1941f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.368+0000 7fd199200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd17c004800 0x7fd1941f57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48840/0 (socket says 172.21.15.26:48840) 2024-04-26T21:09:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.368+0000 7fd199200640 1 -- 172.21.15.26:0/128255036 learned_addr learned my addr 172.21.15.26:0/128255036 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.369+0000 7fd199c00640 1 --2- 172.21.15.26:0/128255036 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd19415fdc0 0x7fd194143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.369+0000 7fd199200640 1 -- 172.21.15.26:0/128255036 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd19415fdc0 msgr2=0x7fd194143b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.369+0000 7fd199200640 1 --2- 172.21.15.26:0/128255036 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd19415fdc0 0x7fd194143b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.369+0000 7fd193e00640 1 --2- 172.21.15.26:0/128255036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd19415d740 0x7fd1941f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.369+0000 7fd199200640 1 -- 172.21.15.26:0/128255036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd19415d740 msgr2=0x7fd1941f2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.369+0000 7fd199200640 1 --2- 172.21.15.26:0/128255036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd19415d740 0x7fd1941f2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.369+0000 7fd199200640 1 -- 172.21.15.26:0/128255036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1941f4cf0 con 0x7fd17c004800 2024-04-26T21:09:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.369+0000 7fd193e00640 1 --2- 172.21.15.26:0/128255036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd19415d740 0x7fd1941f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.369+0000 7fd199200640 1 --2- 172.21.15.26:0/128255036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd17c004800 0x7fd1941f57b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd18800d4b0 tx=0x7fd18800d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:34.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.370+0000 7fd190c00640 1 -- 172.21.15.26:0/128255036 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188005480 con 0x7fd17c004800 2024-04-26T21:09:34.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.370+0000 7fd190c00640 1 -- 172.21.15.26:0/128255036 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd188017070 con 0x7fd17c004800 2024-04-26T21:09:34.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.370+0000 7fd19a600640 1 -- 172.21.15.26:0/128255036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19415c670 con 0x7fd17c004800 2024-04-26T21:09:34.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.370+0000 7fd190c00640 1 -- 172.21.15.26:0/128255036 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188003810 con 0x7fd17c004800 2024-04-26T21:09:34.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.370+0000 7fd19a600640 1 -- 172.21.15.26:0/128255036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1941f4fb0 con 0x7fd17c004800 2024-04-26T21:09:34.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.371+0000 7fd19a600640 1 -- 172.21.15.26:0/128255036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd19406a700 con 0x7fd17c004800 2024-04-26T21:09:34.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.372+0000 7fd190c00640 1 -- 172.21.15.26:0/128255036 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd1880039b0 con 0x7fd17c004800 2024-04-26T21:09:34.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.373+0000 7fd190c00640 1 --2- 172.21.15.26:0/128255036 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd17406c8f0 0x7fd17406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:34.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.373+0000 7fd190c00640 1 -- 172.21.15.26:0/128255036 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fd18808b980 con 0x7fd17c004800 2024-04-26T21:09:34.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.373+0000 7fd193e00640 1 --2- 172.21.15.26:0/128255036 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd17406c8f0 0x7fd17406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:34.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.374+0000 7fd193e00640 1 --2- 172.21.15.26:0/128255036 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd17406c8f0 0x7fd17406edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd180033000 tx=0x7fd180008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:34.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.377+0000 7fd190c00640 1 -- 172.21.15.26:0/128255036 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd188059d50 con 0x7fd17c004800 2024-04-26T21:09:34.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.619+0000 7fd19a600640 1 -- 172.21.15.26:0/128255036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd194070150 con 0x7fd17c004800 2024-04-26T21:09:34.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.620+0000 7fd190c00640 1 -- 172.21.15.26:0/128255036 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7fd18808b070 con 0x7fd17c004800 2024-04-26T21:09:34.621 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:34.621 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/2695371801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2695371801},{"type":"v1","addr":"172.21.15.26:6836","nonce":2695371801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3349732696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3349732696},{"type":"v1","addr":"172.21.15.26:6837","nonce":3349732696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/2288633870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2288633870},{"type":"v1","addr":"172.21.15.86:6833","nonce":2288633870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:08:37.622844+0000","modified":"2024-04-26T21:08:40.715140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/2125360615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2125360615},{"type":"v1","addr":"172.21.15.86:6835","nonce":2125360615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4261,"qdb_cluster":[4261]},"id":1}]} 2024-04-26T21:09:34.622 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2024-04-26T21:09:34.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.623+0000 7fd19a600640 1 -- 172.21.15.26:0/128255036 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd17406c8f0 msgr2=0x7fd17406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:34.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.623+0000 7fd19a600640 1 --2- 172.21.15.26:0/128255036 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd17406c8f0 0x7fd17406edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd180033000 tx=0x7fd180008030 comp rx=0 tx=0).stop 2024-04-26T21:09:34.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.623+0000 7fd19a600640 1 -- 172.21.15.26:0/128255036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd17c004800 msgr2=0x7fd1941f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:34.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.623+0000 7fd19a600640 1 --2- 172.21.15.26:0/128255036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd17c004800 0x7fd1941f57b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd18800d4b0 tx=0x7fd18800d970 comp rx=0 tx=0).stop 2024-04-26T21:09:34.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.623+0000 7fd19a600640 1 -- 172.21.15.26:0/128255036 shutdown_connections 2024-04-26T21:09:34.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.623+0000 7fd19a600640 1 --2- 172.21.15.26:0/128255036 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd17406c8f0 0x7fd17406edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.623+0000 7fd19a600640 1 --2- 172.21.15.26:0/128255036 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd19415fdc0 0x7fd194143b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.623+0000 7fd19a600640 1 --2- 172.21.15.26:0/128255036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd19415d740 0x7fd1941f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.623+0000 7fd19a600640 1 --2- 172.21.15.26:0/128255036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd17c004800 0x7fd1941f57b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.623+0000 7fd19a600640 1 -- 172.21.15.26:0/128255036 >> 172.21.15.26:0/128255036 conn(0x7fd19406c690 msgr2=0x7fd19406ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:34.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.623+0000 7fd19a600640 1 -- 172.21.15.26:0/128255036 shutdown_connections 2024-04-26T21:09:34.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.623+0000 7fd19a600640 1 -- 172.21.15.26:0/128255036 wait complete. 2024-04-26T21:09:34.645 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:09:34.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.859+0000 7face2000640 1 Processor -- start 2024-04-26T21:09:34.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.859+0000 7face2000640 1 -- start start 2024-04-26T21:09:34.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.861+0000 7face2000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7facdc159770 0x7facdc0b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:34.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.861+0000 7face2000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facdc160440 con 0x7facdc15ffa0 2024-04-26T21:09:34.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.861+0000 7face2000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facdc160720 con 0x7facdc159b50 2024-04-26T21:09:34.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.861+0000 7face2000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7facdc072560 con 0x7facdc159770 2024-04-26T21:09:34.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.861+0000 7face0c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7facdc159770 0x7facdc0b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:34.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.861+0000 7face1600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7facdc15ffa0 0x7facdc0d1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60850/0 (socket says 172.21.15.26:60850) 2024-04-26T21:09:34.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.861+0000 7face1600640 1 -- 172.21.15.26:0/2174924865 learned_addr learned my addr 172.21.15.26:0/2174924865 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:34.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.861+0000 7face0c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7facdc159770 0x7facdc0b4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48848/0 (socket says 172.21.15.26:48848) 2024-04-26T21:09:34.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.862+0000 7face0c00640 1 -- 172.21.15.26:0/2174924865 >> v1:172.21.15.86:6789/0 conn(0x7facdc159b50 legacy=0x7facdc15f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:34.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.862+0000 7face0c00640 1 -- 172.21.15.26:0/2174924865 >> v1:172.21.15.26:6789/0 conn(0x7facdc15ffa0 legacy=0x7facdc0d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:34.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.862+0000 7face0c00640 1 -- 172.21.15.26:0/2174924865 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc072250 con 0x7facdc159770 2024-04-26T21:09:34.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.863+0000 7face0c00640 1 --2- 172.21.15.26:0/2174924865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facdc159770 0x7facdc0b4160 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7facd00097d0 tx=0x7facd002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab357f895dbcf70e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:34.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.863+0000 7facdb400640 1 -- 172.21.15.26:0/2174924865 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd0037070 con 0x7facdc159770 2024-04-26T21:09:34.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.863+0000 7facdb400640 1 -- 172.21.15.26:0/2174924865 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd0032440 con 0x7facdc159770 2024-04-26T21:09:34.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.863+0000 7facdb400640 1 -- 172.21.15.26:0/2174924865 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd0032810 con 0x7facdc159770 2024-04-26T21:09:34.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.864+0000 7face2000640 1 -- 172.21.15.26:0/2174924865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facdc159770 msgr2=0x7facdc0b4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:34.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.864+0000 7face2000640 1 --2- 172.21.15.26:0/2174924865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facdc159770 0x7facdc0b4160 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7facd00097d0 tx=0x7facd002a180 comp rx=0 tx=0).stop 2024-04-26T21:09:34.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.864+0000 7face2000640 1 -- 172.21.15.26:0/2174924865 shutdown_connections 2024-04-26T21:09:34.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.864+0000 7face2000640 1 --2- 172.21.15.26:0/2174924865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facdc159770 0x7facdc0b4160 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.864+0000 7face2000640 1 -- 172.21.15.26:0/2174924865 >> 172.21.15.26:0/2174924865 conn(0x7facdc06c690 msgr2=0x7facdc06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:34.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.864+0000 7face2000640 1 -- 172.21.15.26:0/2174924865 shutdown_connections 2024-04-26T21:09:34.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.864+0000 7face2000640 1 -- 172.21.15.26:0/2174924865 wait complete. 2024-04-26T21:09:34.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.866+0000 7face2000640 1 Processor -- start 2024-04-26T21:09:34.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.866+0000 7face2000640 1 -- start start 2024-04-26T21:09:34.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.867+0000 7face2000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facdc159770 0x7facdc1f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:34.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.867+0000 7face2000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facdc159b50 0x7facdc155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:34.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.867+0000 7face2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facdc15ffa0 0x7facdc152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:34.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.867+0000 7face2000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7facdc0d1840 con 0x7facdc159b50 2024-04-26T21:09:34.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.867+0000 7face2000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7facdc06ee10 con 0x7facdc15ffa0 2024-04-26T21:09:34.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.867+0000 7face2000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7facdc0d19c0 con 0x7facdc159770 2024-04-26T21:09:34.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.867+0000 7facdbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facdc159b50 0x7facdc155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:34.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.867+0000 7facdbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facdc159b50 0x7facdc155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48864/0 (socket says 172.21.15.26:48864) 2024-04-26T21:09:34.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.867+0000 7facdbe00640 1 -- 172.21.15.26:0/1020438739 learned_addr learned my addr 172.21.15.26:0/1020438739 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:34.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.868+0000 7face0c00640 1 --2- 172.21.15.26:0/1020438739 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facdc159770 0x7facdc1f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:34.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.868+0000 7face1600640 1 --2- 172.21.15.26:0/1020438739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facdc15ffa0 0x7facdc152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:34.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.868+0000 7facdbe00640 1 -- 172.21.15.26:0/1020438739 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facdc159770 msgr2=0x7facdc1f5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:34.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.868+0000 7facdbe00640 1 --2- 172.21.15.26:0/1020438739 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facdc159770 0x7facdc1f5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.868+0000 7facdbe00640 1 -- 172.21.15.26:0/1020438739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facdc15ffa0 msgr2=0x7facdc152600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:34.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.868+0000 7facdbe00640 1 --2- 172.21.15.26:0/1020438739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facdc15ffa0 0x7facdc152600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:34.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.868+0000 7facdbe00640 1 -- 172.21.15.26:0/1020438739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc1f5c40 con 0x7facdc159b50 2024-04-26T21:09:34.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.868+0000 7facdbe00640 1 --2- 172.21.15.26:0/1020438739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facdc159b50 0x7facdc155a70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7facc800b3c0 tx=0x7facc800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:34.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.868+0000 7facd8c00640 1 -- 172.21.15.26:0/1020438739 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8011830 con 0x7facdc159b50 2024-04-26T21:09:34.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.868+0000 7facd8c00640 1 -- 172.21.15.26:0/1020438739 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc8016070 con 0x7facdc159b50 2024-04-26T21:09:34.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.868+0000 7face2000640 1 -- 172.21.15.26:0/1020438739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facdc155fb0 con 0x7facdc159b50 2024-04-26T21:09:34.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.869+0000 7facd8c00640 1 -- 172.21.15.26:0/1020438739 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8011c50 con 0x7facdc159b50 2024-04-26T21:09:34.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.869+0000 7face2000640 1 -- 172.21.15.26:0/1020438739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facdc154cb0 con 0x7facdc159b50 2024-04-26T21:09:34.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.870+0000 7face2000640 1 -- 172.21.15.26:0/1020438739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facdc06a700 con 0x7facdc159b50 2024-04-26T21:09:34.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.871+0000 7facd8c00640 1 -- 172.21.15.26:0/1020438739 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7facc801b460 con 0x7facdc159b50 2024-04-26T21:09:34.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.872+0000 7facd8c00640 1 --2- 172.21.15.26:0/1020438739 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7facbc06c8f0 0x7facbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:34.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.872+0000 7facd8c00640 1 -- 172.21.15.26:0/1020438739 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7facc808b700 con 0x7facdc159b50 2024-04-26T21:09:34.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.872+0000 7face0c00640 1 --2- 172.21.15.26:0/1020438739 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7facbc06c8f0 0x7facbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:34.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.873+0000 7face0c00640 1 --2- 172.21.15.26:0/1020438739 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7facbc06c8f0 0x7facbc06edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7facd0034000 tx=0x7facd002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:34.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:34.876+0000 7facd8c00640 1 -- 172.21.15.26:0/1020438739 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7facc8020020 con 0x7facdc159b50 2024-04-26T21:09:35.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.080+0000 7face2000640 1 -- 172.21.15.26:0/1020438739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7facdc06d250 con 0x7facdc159b50 2024-04-26T21:09:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.083+0000 7facd8c00640 1 -- 172.21.15.26:0/1020438739 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16806 (secure 0 0 0) 0x7facdc1f5c40 con 0x7facdc159b50 2024-04-26T21:09:35.084 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:35.084 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:09:24.561115+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:08:35.481169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:08:36.609498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:09:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.085+0000 7face2000640 1 -- 172.21.15.26:0/1020438739 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7facbc06c8f0 msgr2=0x7facbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.085+0000 7face2000640 1 --2- 172.21.15.26:0/1020438739 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7facbc06c8f0 0x7facbc06edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7facd0034000 tx=0x7facd002e000 comp rx=0 tx=0).stop 2024-04-26T21:09:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.085+0000 7face2000640 1 -- 172.21.15.26:0/1020438739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facdc159b50 msgr2=0x7facdc155a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.085+0000 7face2000640 1 --2- 172.21.15.26:0/1020438739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facdc159b50 0x7facdc155a70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7facc800b3c0 tx=0x7facc800b880 comp rx=0 tx=0).stop 2024-04-26T21:09:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.085+0000 7face2000640 1 -- 172.21.15.26:0/1020438739 shutdown_connections 2024-04-26T21:09:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.085+0000 7face2000640 1 --2- 172.21.15.26:0/1020438739 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7facbc06c8f0 0x7facbc06edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.085+0000 7face2000640 1 --2- 172.21.15.26:0/1020438739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facdc15ffa0 0x7facdc152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.085+0000 7face2000640 1 --2- 172.21.15.26:0/1020438739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facdc159b50 0x7facdc155a70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.085+0000 7face2000640 1 --2- 172.21.15.26:0/1020438739 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facdc159770 0x7facdc1f5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.085+0000 7face2000640 1 -- 172.21.15.26:0/1020438739 >> 172.21.15.26:0/1020438739 conn(0x7facdc06c690 msgr2=0x7facdc0b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.086+0000 7face2000640 1 -- 172.21.15.26:0/1020438739 shutdown_connections 2024-04-26T21:09:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.086+0000 7face2000640 1 -- 172.21.15.26:0/1020438739 wait complete. 2024-04-26T21:09:35.105 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:09:35.105 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:35.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.322+0000 7f464d000640 1 Processor -- start 2024-04-26T21:09:35.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.322+0000 7f464d000640 1 -- start start 2024-04-26T21:09:35.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.323+0000 7f464d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4648162ba0 0x7f46480d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:35.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.323+0000 7f464d000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f46480d2060 0x7f46480d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:35.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.323+0000 7f464d000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46480b61c0 con 0x7f46480d1c80 2024-04-26T21:09:35.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.323+0000 7f464d000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f46480b5f70 con 0x7f46480d2060 2024-04-26T21:09:35.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.323+0000 7f464d000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f46480d3830 con 0x7f4648162ba0 2024-04-26T21:09:35.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.323+0000 7f4647e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f46480d2060 0x7f46480d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:35.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.324+0000 7f4647e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f46480d2060 0x7f46480d3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48874/0 (socket says 172.21.15.26:48874) 2024-04-26T21:09:35.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.324+0000 7f4647e00640 1 -- 172.21.15.26:0/4076374840 learned_addr learned my addr 172.21.15.26:0/4076374840 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:35.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.324+0000 7f463fe00640 1 --2- 172.21.15.26:0/4076374840 >> v2:172.21.15.86:3300/0 conn(0x7f4648162ba0 0x7f46480d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:35.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.324+0000 7f4647e00640 1 -- 172.21.15.26:0/4076374840 >> v2:172.21.15.86:3300/0 conn(0x7f4648162ba0 msgr2=0x7f46480d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:35.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.324+0000 7f4647e00640 1 --2- 172.21.15.26:0/4076374840 >> v2:172.21.15.86:3300/0 conn(0x7f4648162ba0 0x7f46480d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:35.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.324+0000 7f4647e00640 1 -- 172.21.15.26:0/4076374840 >> v1:172.21.15.86:6789/0 conn(0x7f46480d1c80 legacy=0x7f4648162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:35.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.324+0000 7f4647e00640 1 -- 172.21.15.26:0/4076374840 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46480b64a0 con 0x7f46480d2060 2024-04-26T21:09:35.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.324+0000 7f463fe00640 1 --2- 172.21.15.26:0/4076374840 >> v2:172.21.15.86:3300/0 conn(0x7f4648162ba0 0x7f46480d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:09:35.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.325+0000 7f4647e00640 1 --2- 172.21.15.26:0/4076374840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46480d2060 0x7f46480d3cc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f46480d4ff0 tx=0x7f464002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba135c47b1d764b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:35.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.325+0000 7f4646a00640 1 -- 172.21.15.26:0/4076374840 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640036070 con 0x7f46480d2060 2024-04-26T21:09:35.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.326+0000 7f4646a00640 1 -- 172.21.15.26:0/4076374840 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4640031600 con 0x7f46480d2060 2024-04-26T21:09:35.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.326+0000 7f4646a00640 1 -- 172.21.15.26:0/4076374840 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46400319d0 con 0x7f46480d2060 2024-04-26T21:09:35.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.326+0000 7f464d000640 1 -- 172.21.15.26:0/4076374840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46480d2060 msgr2=0x7f46480d3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:35.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.326+0000 7f464d000640 1 --2- 172.21.15.26:0/4076374840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46480d2060 0x7f46480d3cc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f46480d4ff0 tx=0x7f464002a660 comp rx=0 tx=0).stop 2024-04-26T21:09:35.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.326+0000 7f464d000640 1 -- 172.21.15.26:0/4076374840 shutdown_connections 2024-04-26T21:09:35.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.326+0000 7f464d000640 1 --2- 172.21.15.26:0/4076374840 >> v2:172.21.15.86:3300/0 conn(0x7f4648162ba0 0x7f46480d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:35.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.326+0000 7f464d000640 1 --2- 172.21.15.26:0/4076374840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46480d2060 0x7f46480d3cc0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:35.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.326+0000 7f464d000640 1 -- 172.21.15.26:0/4076374840 >> 172.21.15.26:0/4076374840 conn(0x7f46480d5020 msgr2=0x7f46480d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:35.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.326+0000 7f464d000640 1 -- 172.21.15.26:0/4076374840 shutdown_connections 2024-04-26T21:09:35.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.326+0000 7f464d000640 1 -- 172.21.15.26:0/4076374840 wait complete. 2024-04-26T21:09:35.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.328+0000 7f464d000640 1 Processor -- start 2024-04-26T21:09:35.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.328+0000 7f464d000640 1 -- start start 2024-04-26T21:09:35.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.329+0000 7f464d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46480d1c80 0x7f46480b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:35.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.329+0000 7f464d000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46480d2060 0x7f464815cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:35.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.329+0000 7f464d000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4648162ba0 0x7f46480cd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:35.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.329+0000 7f464d000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f46480d5cc0 con 0x7f4648162ba0 2024-04-26T21:09:35.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.329+0000 7f464d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46480d5b40 con 0x7f46480d1c80 2024-04-26T21:09:35.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.329+0000 7f464d000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f46480c8b70 con 0x7f46480d2060 2024-04-26T21:09:35.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.330+0000 7f4647e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4648162ba0 0x7f46480cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:35.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.330+0000 7f4647e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4648162ba0 0x7f46480cd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48882/0 (socket says 172.21.15.26:48882) 2024-04-26T21:09:35.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.330+0000 7f4647e00640 1 -- 172.21.15.26:0/3472238892 learned_addr learned my addr 172.21.15.26:0/3472238892 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:35.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.330+0000 7f463fe00640 1 --2- 172.21.15.26:0/3472238892 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46480d2060 0x7f464815cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:35.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.330+0000 7f4647e00640 1 -- 172.21.15.26:0/3472238892 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46480d2060 msgr2=0x7f464815cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:35.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.330+0000 7f4647e00640 1 --2- 172.21.15.26:0/3472238892 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46480d2060 0x7f464815cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:35.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.330+0000 7f4647e00640 1 -- 172.21.15.26:0/3472238892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46480d1c80 msgr2=0x7f46480b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:35.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.330+0000 7f4647400640 1 --2- 172.21.15.26:0/3472238892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46480d1c80 0x7f46480b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:35.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.330+0000 7f4647e00640 1 --2- 172.21.15.26:0/3472238892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46480d1c80 0x7f46480b1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:35.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.330+0000 7f4647e00640 1 -- 172.21.15.26:0/3472238892 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46480b9ec0 con 0x7f4648162ba0 2024-04-26T21:09:35.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.330+0000 7f4647400640 1 --2- 172.21.15.26:0/3472238892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46480d1c80 0x7f46480b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:35.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.330+0000 7f4647e00640 1 --2- 172.21.15.26:0/3472238892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4648162ba0 0x7f46480cd720 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f464002ac50 tx=0x7f4640004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:35.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.331+0000 7f463f400640 1 -- 172.21.15.26:0/3472238892 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640041030 con 0x7f4648162ba0 2024-04-26T21:09:35.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.331+0000 7f463f400640 1 -- 172.21.15.26:0/3472238892 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4640031400 con 0x7f4648162ba0 2024-04-26T21:09:35.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.331+0000 7f463f400640 1 -- 172.21.15.26:0/3472238892 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640036040 con 0x7f4648162ba0 2024-04-26T21:09:35.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.331+0000 7f464d000640 1 -- 172.21.15.26:0/3472238892 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46480ba1a0 con 0x7f4648162ba0 2024-04-26T21:09:35.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.331+0000 7f464d000640 1 -- 172.21.15.26:0/3472238892 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46480b2170 con 0x7f4648162ba0 2024-04-26T21:09:35.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.332+0000 7f464d000640 1 -- 172.21.15.26:0/3472238892 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f464806a700 con 0x7f4648162ba0 2024-04-26T21:09:35.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.333+0000 7f463f400640 1 -- 172.21.15.26:0/3472238892 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4640044070 con 0x7f4648162ba0 2024-04-26T21:09:35.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.334+0000 7f463f400640 1 --2- 172.21.15.26:0/3472238892 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f462406c920 0x7f462406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:35.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.334+0000 7f463f400640 1 -- 172.21.15.26:0/3472238892 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f46400ac560 con 0x7f4648162ba0 2024-04-26T21:09:35.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.334+0000 7f4647400640 1 --2- 172.21.15.26:0/3472238892 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f462406c920 0x7f462406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:35.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.335+0000 7f4647400640 1 --2- 172.21.15.26:0/3472238892 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f462406c920 0x7f462406ede0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4638003eb0 tx=0x7f4638009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:35.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.339+0000 7f463f400640 1 -- 172.21.15.26:0/3472238892 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f464007a900 con 0x7f4648162ba0 2024-04-26T21:09:35.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.584+0000 7f464d000640 1 -- 172.21.15.26:0/3472238892 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46480d3ef0 con 0x7f4648162ba0 2024-04-26T21:09:35.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.585+0000 7f463f400640 1 -- 172.21.15.26:0/3472238892 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7f464007a200 con 0x7f4648162ba0 2024-04-26T21:09:35.586 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:35.586 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/2695371801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2695371801},{"type":"v1","addr":"172.21.15.26:6836","nonce":2695371801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3349732696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3349732696},{"type":"v1","addr":"172.21.15.26:6837","nonce":3349732696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/2288633870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2288633870},{"type":"v1","addr":"172.21.15.86:6833","nonce":2288633870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:08:37.622844+0000","modified":"2024-04-26T21:08:40.715140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/2125360615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2125360615},{"type":"v1","addr":"172.21.15.86:6835","nonce":2125360615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4261,"qdb_cluster":[4261]},"id":1}]} 2024-04-26T21:09:35.586 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2024-04-26T21:09:35.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.587+0000 7f464d000640 1 -- 172.21.15.26:0/3472238892 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f462406c920 msgr2=0x7f462406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:35.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.587+0000 7f464d000640 1 --2- 172.21.15.26:0/3472238892 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f462406c920 0x7f462406ede0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4638003eb0 tx=0x7f4638009040 comp rx=0 tx=0).stop 2024-04-26T21:09:35.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.587+0000 7f464d000640 1 -- 172.21.15.26:0/3472238892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4648162ba0 msgr2=0x7f46480cd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:35.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.587+0000 7f464d000640 1 --2- 172.21.15.26:0/3472238892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4648162ba0 0x7f46480cd720 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f464002ac50 tx=0x7f4640004a40 comp rx=0 tx=0).stop 2024-04-26T21:09:35.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.588+0000 7f464d000640 1 -- 172.21.15.26:0/3472238892 shutdown_connections 2024-04-26T21:09:35.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.588+0000 7f464d000640 1 --2- 172.21.15.26:0/3472238892 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f462406c920 0x7f462406ede0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:35.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.588+0000 7f464d000640 1 --2- 172.21.15.26:0/3472238892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4648162ba0 0x7f46480cd720 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:35.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.588+0000 7f464d000640 1 --2- 172.21.15.26:0/3472238892 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46480d2060 0x7f464815cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:35.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.588+0000 7f464d000640 1 --2- 172.21.15.26:0/3472238892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46480d1c80 0x7f46480b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:35.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.588+0000 7f464d000640 1 -- 172.21.15.26:0/3472238892 >> 172.21.15.26:0/3472238892 conn(0x7f46480d5020 msgr2=0x7f46480caf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:35.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.588+0000 7f464d000640 1 -- 172.21.15.26:0/3472238892 shutdown_connections 2024-04-26T21:09:35.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.588+0000 7f464d000640 1 -- 172.21.15.26:0/3472238892 wait complete. 2024-04-26T21:09:35.608 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:35.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.817+0000 7f5d25800640 1 Processor -- start 2024-04-26T21:09:35.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.817+0000 7f5d25800640 1 -- start start 2024-04-26T21:09:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.819+0000 7f5d25800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5d20162570 0x7f5d201629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.819+0000 7f5d25800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d200d3d80 con 0x7f5d200d1d60 2024-04-26T21:09:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.819+0000 7f5d25800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d200cfe90 con 0x7f5d200d1980 2024-04-26T21:09:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.819+0000 7f5d25800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5d200d04e0 con 0x7f5d20162570 2024-04-26T21:09:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.819+0000 7f5d1f400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5d200d1d60 0x7f5d200b7f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60860/0 (socket says 172.21.15.26:60860) 2024-04-26T21:09:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.819+0000 7f5d1f400640 1 -- 172.21.15.26:0/1898114716 learned_addr learned my addr 172.21.15.26:0/1898114716 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.819+0000 7f5d24e00640 1 --2- 172.21.15.26:0/1898114716 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5d20162570 0x7f5d201629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:35.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.820+0000 7f5d1ea00640 1 -- 172.21.15.26:0/1898114716 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 522132664 0 0) 0x7f5d200d3d80 con 0x7f5d200d1d60 2024-04-26T21:09:35.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.820+0000 7f5d1ea00640 1 -- 172.21.15.26:0/1898114716 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d04003520 con 0x7f5d200d1d60 2024-04-26T21:09:35.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.820+0000 7f5d1ea00640 1 -- 172.21.15.26:0/1898114716 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1615244640 0 0) 0x7f5d04003520 con 0x7f5d200d1d60 2024-04-26T21:09:35.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.820+0000 7f5d1ea00640 1 -- 172.21.15.26:0/1898114716 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5d20162570 msgr2=0x7f5d201629c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:35.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.820+0000 7f5d1ea00640 1 --2- 172.21.15.26:0/1898114716 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5d20162570 0x7f5d201629c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:35.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.820+0000 7f5d1ea00640 1 -- 172.21.15.26:0/1898114716 >> v1:172.21.15.86:6789/0 conn(0x7f5d200d1980 legacy=0x7f5d200d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:35.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.820+0000 7f5d1ea00640 1 -- 172.21.15.26:0/1898114716 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d200d0170 con 0x7f5d200d1d60 2024-04-26T21:09:35.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.820+0000 7f5d1ea00640 1 -- 172.21.15.26:0/1898114716 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f5d080019f0 con 0x7f5d200d1d60 2024-04-26T21:09:35.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.820+0000 7f5d1ea00640 1 -- 172.21.15.26:0/1898114716 >> v1:172.21.15.26:6789/0 conn(0x7f5d200d1d60 legacy=0x7f5d200b7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:35.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.820+0000 7f5d1ea00640 1 --2- 172.21.15.26:0/1898114716 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5d20162570 0x7f5d040039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.820+0000 7f5d1ea00640 1 --2- 172.21.15.26:0/1898114716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d04003f60 0x7f5d04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.820+0000 7f5d1ea00640 1 --2- 172.21.15.26:0/1898114716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d04004d40 0x7f5d040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.820+0000 7f5d1ea00640 1 -- 172.21.15.26:0/1898114716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5d200d04e0 con 0x7f5d04004d40 2024-04-26T21:09:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.820+0000 7f5d1ea00640 1 -- 172.21.15.26:0/1898114716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0400baa0 con 0x7f5d04003f60 2024-04-26T21:09:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.820+0000 7f5d1ea00640 1 -- 172.21.15.26:0/1898114716 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5d0400bca0 con 0x7f5d20162570 2024-04-26T21:09:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.821+0000 7f5d24e00640 1 --2- 172.21.15.26:0/1898114716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d04004d40 0x7f5d040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.821+0000 7f5d1fe00640 1 --2- 172.21.15.26:0/1898114716 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5d20162570 0x7f5d040039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.821+0000 7f5d1f400640 1 --2- 172.21.15.26:0/1898114716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d04003f60 0x7f5d04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.821+0000 7f5d24e00640 1 -- 172.21.15.26:0/1898114716 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5d20162570 msgr2=0x7f5d040039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.821+0000 7f5d24e00640 1 --2- 172.21.15.26:0/1898114716 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5d20162570 0x7f5d040039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.821+0000 7f5d24e00640 1 -- 172.21.15.26:0/1898114716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d04003f60 msgr2=0x7f5d04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.821+0000 7f5d24e00640 1 --2- 172.21.15.26:0/1898114716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d04003f60 0x7f5d04004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.821+0000 7f5d24e00640 1 -- 172.21.15.26:0/1898114716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d04003520 con 0x7f5d04004d40 2024-04-26T21:09:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.822+0000 7f5d24e00640 1 --2- 172.21.15.26:0/1898114716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d04004d40 0x7f5d040051c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009ac0 tx=0x7f5d04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5906d9803e0892 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.822+0000 7f5d1ea00640 1 -- 172.21.15.26:0/1898114716 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18036070 con 0x7f5d04004d40 2024-04-26T21:09:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.823+0000 7f5d1ea00640 1 -- 172.21.15.26:0/1898114716 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d18031400 con 0x7f5d04004d40 2024-04-26T21:09:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.823+0000 7f5d1ea00640 1 -- 172.21.15.26:0/1898114716 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18009690 con 0x7f5d04004d40 2024-04-26T21:09:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.823+0000 7f5d25800640 1 -- 172.21.15.26:0/1898114716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d04004d40 msgr2=0x7f5d040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.823+0000 7f5d25800640 1 --2- 172.21.15.26:0/1898114716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d04004d40 0x7f5d040051c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009ac0 tx=0x7f5d04005830 comp rx=0 tx=0).stop 2024-04-26T21:09:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.823+0000 7f5d24e00640 1 -- 172.21.15.26:0/1898114716 reap_dead start 2024-04-26T21:09:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.823+0000 7f5d25800640 1 -- 172.21.15.26:0/1898114716 shutdown_connections 2024-04-26T21:09:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.823+0000 7f5d25800640 1 -- 172.21.15.26:0/1898114716 >> 172.21.15.26:0/1898114716 conn(0x7f5d2009e410 msgr2=0x7f5d200d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.823+0000 7f5d25800640 1 -- 172.21.15.26:0/1898114716 shutdown_connections 2024-04-26T21:09:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.823+0000 7f5d25800640 1 -- 172.21.15.26:0/1898114716 wait complete. 2024-04-26T21:09:35.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.825+0000 7f5d25800640 1 Processor -- start 2024-04-26T21:09:35.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.825+0000 7f5d25800640 1 -- start start 2024-04-26T21:09:35.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.826+0000 7f5d25800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d04004d40 0x7f5d2015be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:35.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.826+0000 7f5d25800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d200cb790 0x7f5d20156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:35.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.826+0000 7f5d25800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5d200ce380 0x7f5d200ce830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:35.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.826+0000 7f5d25800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5d200d41e0 con 0x7f5d04004d40 2024-04-26T21:09:35.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.826+0000 7f5d25800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d200d4060 con 0x7f5d200cb790 2024-04-26T21:09:35.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.826+0000 7f5d25800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5d200b8640 con 0x7f5d200ce380 2024-04-26T21:09:35.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.827+0000 7f5d1fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d04004d40 0x7f5d2015be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:35.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.827+0000 7f5d1fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d04004d40 0x7f5d2015be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48898/0 (socket says 172.21.15.26:48898) 2024-04-26T21:09:35.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.827+0000 7f5d1fe00640 1 -- 172.21.15.26:0/75383876 learned_addr learned my addr 172.21.15.26:0/75383876 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:35.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.827+0000 7f5d24e00640 1 --2- 172.21.15.26:0/75383876 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5d200ce380 0x7f5d200ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:35.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.827+0000 7f5d1f400640 1 --2- 172.21.15.26:0/75383876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d200cb790 0x7f5d20156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:35.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.827+0000 7f5d1fe00640 1 -- 172.21.15.26:0/75383876 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5d200ce380 msgr2=0x7f5d200ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:35.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.827+0000 7f5d1fe00640 1 --2- 172.21.15.26:0/75383876 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5d200ce380 0x7f5d200ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:35.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.827+0000 7f5d1fe00640 1 -- 172.21.15.26:0/75383876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d200cb790 msgr2=0x7f5d20156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:35.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.827+0000 7f5d1fe00640 1 --2- 172.21.15.26:0/75383876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d200cb790 0x7f5d20156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:35.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.827+0000 7f5d1fe00640 1 -- 172.21.15.26:0/75383876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2015f170 con 0x7f5d04004d40 2024-04-26T21:09:35.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.827+0000 7f5d1f400640 1 --2- 172.21.15.26:0/75383876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d200cb790 0x7f5d20156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:35.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.828+0000 7f5d1fe00640 1 --2- 172.21.15.26:0/75383876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d04004d40 0x7f5d2015be00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000e550 tx=0x7f5d1000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:35.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.828+0000 7f5d17e00640 1 -- 172.21.15.26:0/75383876 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d100052e0 con 0x7f5d04004d40 2024-04-26T21:09:35.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.828+0000 7f5d17e00640 1 -- 172.21.15.26:0/75383876 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d10018070 con 0x7f5d04004d40 2024-04-26T21:09:35.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.828+0000 7f5d17e00640 1 -- 172.21.15.26:0/75383876 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d10003810 con 0x7f5d04004d40 2024-04-26T21:09:35.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.828+0000 7f5d25800640 1 -- 172.21.15.26:0/75383876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d200cbf70 con 0x7f5d04004d40 2024-04-26T21:09:35.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.828+0000 7f5d25800640 1 -- 172.21.15.26:0/75383876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d2015e690 con 0x7f5d04004d40 2024-04-26T21:09:35.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.829+0000 7f5d25800640 1 -- 172.21.15.26:0/75383876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d200693d0 con 0x7f5d04004d40 2024-04-26T21:09:35.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.830+0000 7f5d17e00640 1 -- 172.21.15.26:0/75383876 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d100039b0 con 0x7f5d04004d40 2024-04-26T21:09:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.831+0000 7f5d17e00640 1 --2- 172.21.15.26:0/75383876 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5cfc06c920 0x7f5cfc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.831+0000 7f5d17e00640 1 -- 172.21.15.26:0/75383876 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f5d1008b8c0 con 0x7f5d04004d40 2024-04-26T21:09:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.832+0000 7f5d1f400640 1 --2- 172.21.15.26:0/75383876 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5cfc06c920 0x7f5cfc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.832+0000 7f5d1f400640 1 --2- 172.21.15.26:0/75383876 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5cfc06c920 0x7f5cfc06ede0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5d08001580 tx=0x7f5d080014d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:35.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:35.836+0000 7f5d17e00640 1 -- 172.21.15.26:0/75383876 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5d10059bd0 con 0x7f5d04004d40 2024-04-26T21:09:36.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.078+0000 7f5d25800640 1 -- 172.21.15.26:0/75383876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d200b6b50 con 0x7f5d04004d40 2024-04-26T21:09:36.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.080+0000 7f5d17e00640 1 -- 172.21.15.26:0/75383876 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7f5d100594d0 con 0x7f5d04004d40 2024-04-26T21:09:36.080 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:36.080 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/2695371801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2695371801},{"type":"v1","addr":"172.21.15.26:6836","nonce":2695371801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3349732696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3349732696},{"type":"v1","addr":"172.21.15.26:6837","nonce":3349732696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6833/2288633870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2288633870},{"type":"v1","addr":"172.21.15.86:6833","nonce":2288633870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:08:37.622844+0000","modified":"2024-04-26T21:08:40.715140+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6835/2125360615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2125360615},{"type":"v1","addr":"172.21.15.86:6835","nonce":2125360615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4261,"qdb_cluster":[4261]},"id":1}]} 2024-04-26T21:09:36.081 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2024-04-26T21:09:36.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.082+0000 7f5d25800640 1 -- 172.21.15.26:0/75383876 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5cfc06c920 msgr2=0x7f5cfc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.082+0000 7f5d25800640 1 --2- 172.21.15.26:0/75383876 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5cfc06c920 0x7f5cfc06ede0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5d08001580 tx=0x7f5d080014d0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.082+0000 7f5d25800640 1 -- 172.21.15.26:0/75383876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d04004d40 msgr2=0x7f5d2015be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.082+0000 7f5d25800640 1 --2- 172.21.15.26:0/75383876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d04004d40 0x7f5d2015be00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000e550 tx=0x7f5d1000ea10 comp rx=0 tx=0).stop 2024-04-26T21:09:36.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.082+0000 7f5d25800640 1 -- 172.21.15.26:0/75383876 shutdown_connections 2024-04-26T21:09:36.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.082+0000 7f5d25800640 1 --2- 172.21.15.26:0/75383876 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5cfc06c920 0x7f5cfc06ede0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.082+0000 7f5d25800640 1 --2- 172.21.15.26:0/75383876 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5d200ce380 0x7f5d200ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.082+0000 7f5d25800640 1 --2- 172.21.15.26:0/75383876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d200cb790 0x7f5d20156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.082+0000 7f5d25800640 1 --2- 172.21.15.26:0/75383876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d04004d40 0x7f5d2015be00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.082+0000 7f5d25800640 1 -- 172.21.15.26:0/75383876 >> 172.21.15.26:0/75383876 conn(0x7f5d2009e410 msgr2=0x7f5d200d2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:36.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.082+0000 7f5d25800640 1 -- 172.21.15.26:0/75383876 shutdown_connections 2024-04-26T21:09:36.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.082+0000 7f5d25800640 1 -- 172.21.15.26:0/75383876 wait complete. 2024-04-26T21:09:36.101 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:09:36.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.308+0000 7fea64a00640 1 Processor -- start 2024-04-26T21:09:36.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.309+0000 7fea64a00640 1 -- start start 2024-04-26T21:09:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.310+0000 7fea64a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fea60073470 0x7fea60073850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.310+0000 7fea64a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fea6006f340 0x7fea6015faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.311+0000 7fea64a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea6006f7d0 con 0x7fea6015ffe0 2024-04-26T21:09:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.311+0000 7fea64a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fea6006fab0 con 0x7fea6006f340 2024-04-26T21:09:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.311+0000 7fea64a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fea6006fd90 con 0x7fea60073470 2024-04-26T21:09:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.311+0000 7fea57e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fea6006f340 0x7fea6015faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.311+0000 7fea5fe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fea6015ffe0 0x7fea600d11f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60866/0 (socket says 172.21.15.26:60866) 2024-04-26T21:09:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.311+0000 7fea5fe00640 1 -- 172.21.15.26:0/3878677745 learned_addr learned my addr 172.21.15.26:0/3878677745 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:36.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.311+0000 7fea57e00640 1 -- 172.21.15.26:0/3878677745 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fea60073470 msgr2=0x7fea60073850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.311+0000 7fea5f400640 1 --2- 172.21.15.26:0/3878677745 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fea60073470 0x7fea60073850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.311+0000 7fea57e00640 1 --2- 172.21.15.26:0/3878677745 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fea60073470 0x7fea60073850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.311+0000 7fea5ea00640 1 -- 172.21.15.26:0/3878677745 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1463731972 0 0) 0x7fea6006f7d0 con 0x7fea6015ffe0 2024-04-26T21:09:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.311+0000 7fea57e00640 1 -- 172.21.15.26:0/3878677745 >> v1:172.21.15.26:6789/0 conn(0x7fea6015ffe0 legacy=0x7fea600d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.311+0000 7fea57e00640 1 -- 172.21.15.26:0/3878677745 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea60070070 con 0x7fea6006f340 2024-04-26T21:09:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.311+0000 7fea5f400640 1 --2- 172.21.15.26:0/3878677745 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fea60073470 0x7fea60073850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.313+0000 7fea57e00640 1 --2- 172.21.15.26:0/3878677745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea6006f340 0x7fea6015faa0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fea480097a0 tx=0x7fea4802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ed01e6967f3a02 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.313+0000 7fea5ea00640 1 -- 172.21.15.26:0/3878677745 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48037070 con 0x7fea6006f340 2024-04-26T21:09:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.313+0000 7fea5ea00640 1 -- 172.21.15.26:0/3878677745 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea48032440 con 0x7fea6006f340 2024-04-26T21:09:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.313+0000 7fea5ea00640 1 -- 172.21.15.26:0/3878677745 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48032810 con 0x7fea6006f340 2024-04-26T21:09:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.313+0000 7fea64a00640 1 -- 172.21.15.26:0/3878677745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea6006f340 msgr2=0x7fea6015faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.313+0000 7fea64a00640 1 --2- 172.21.15.26:0/3878677745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea6006f340 0x7fea6015faa0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fea480097a0 tx=0x7fea4802a180 comp rx=0 tx=0).stop 2024-04-26T21:09:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.314+0000 7fea64a00640 1 -- 172.21.15.26:0/3878677745 shutdown_connections 2024-04-26T21:09:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.314+0000 7fea64a00640 1 --2- 172.21.15.26:0/3878677745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea6006f340 0x7fea6015faa0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.314+0000 7fea64a00640 1 --2- 172.21.15.26:0/3878677745 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fea60073470 0x7fea60073850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.314+0000 7fea64a00640 1 -- 172.21.15.26:0/3878677745 >> 172.21.15.26:0/3878677745 conn(0x7fea6006c690 msgr2=0x7fea6006caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.314+0000 7fea64a00640 1 -- 172.21.15.26:0/3878677745 shutdown_connections 2024-04-26T21:09:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.314+0000 7fea64a00640 1 -- 172.21.15.26:0/3878677745 wait complete. 2024-04-26T21:09:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.315+0000 7fea64a00640 1 Processor -- start 2024-04-26T21:09:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.316+0000 7fea64a00640 1 -- start start 2024-04-26T21:09:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.317+0000 7fea64a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea60073470 0x7fea6015b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.317+0000 7fea64a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fea6015ffe0 0x7fea600b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.317+0000 7fea64a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea600b1390 0x7fea60158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.317+0000 7fea64a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fea60073f10 con 0x7fea600b1390 2024-04-26T21:09:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.317+0000 7fea64a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea60073d90 con 0x7fea60073470 2024-04-26T21:09:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.317+0000 7fea64a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fea600d1900 con 0x7fea6015ffe0 2024-04-26T21:09:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.317+0000 7fea5fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea600b1390 0x7fea60158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.317+0000 7fea5fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea600b1390 0x7fea60158980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48920/0 (socket says 172.21.15.26:48920) 2024-04-26T21:09:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.317+0000 7fea5fe00640 1 -- 172.21.15.26:0/4010103808 learned_addr learned my addr 172.21.15.26:0/4010103808 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.317+0000 7fea57e00640 1 --2- 172.21.15.26:0/4010103808 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fea6015ffe0 0x7fea600b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.317+0000 7fea5f400640 1 --2- 172.21.15.26:0/4010103808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea60073470 0x7fea6015b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.317+0000 7fea5fe00640 1 -- 172.21.15.26:0/4010103808 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fea6015ffe0 msgr2=0x7fea600b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.317+0000 7fea5fe00640 1 --2- 172.21.15.26:0/4010103808 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fea6015ffe0 0x7fea600b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.317+0000 7fea5fe00640 1 -- 172.21.15.26:0/4010103808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea60073470 msgr2=0x7fea6015b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.317+0000 7fea5fe00640 1 --2- 172.21.15.26:0/4010103808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea60073470 0x7fea6015b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.317+0000 7fea5fe00640 1 -- 172.21.15.26:0/4010103808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea600af710 con 0x7fea600b1390 2024-04-26T21:09:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.318+0000 7fea5fe00640 1 --2- 172.21.15.26:0/4010103808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea600b1390 0x7fea60158980 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fea5800c700 tx=0x7fea5800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.318+0000 7fea57400640 1 -- 172.21.15.26:0/4010103808 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58018070 con 0x7fea600b1390 2024-04-26T21:09:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.318+0000 7fea57400640 1 -- 172.21.15.26:0/4010103808 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea5800ce10 con 0x7fea600b1390 2024-04-26T21:09:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.318+0000 7fea57400640 1 -- 172.21.15.26:0/4010103808 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5800a880 con 0x7fea600b1390 2024-04-26T21:09:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.318+0000 7fea64a00640 1 -- 172.21.15.26:0/4010103808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea60157bc0 con 0x7fea600b1390 2024-04-26T21:09:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.318+0000 7fea64a00640 1 -- 172.21.15.26:0/4010103808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea60158180 con 0x7fea600b1390 2024-04-26T21:09:36.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.319+0000 7fea64a00640 1 -- 172.21.15.26:0/4010103808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea6006a700 con 0x7fea600b1390 2024-04-26T21:09:36.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.321+0000 7fea57400640 1 -- 172.21.15.26:0/4010103808 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea5801b070 con 0x7fea600b1390 2024-04-26T21:09:36.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.321+0000 7fea57400640 1 --2- 172.21.15.26:0/4010103808 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fea3c06c940 0x7fea3c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:36.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.322+0000 7fea57400640 1 -- 172.21.15.26:0/4010103808 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fea5808b280 con 0x7fea600b1390 2024-04-26T21:09:36.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.322+0000 7fea5f400640 1 --2- 172.21.15.26:0/4010103808 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fea3c06c940 0x7fea3c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:36.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.323+0000 7fea5f400640 1 --2- 172.21.15.26:0/4010103808 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fea3c06c940 0x7fea3c06ee00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fea5000a990 tx=0x7fea50008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:36.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.326+0000 7fea57400640 1 -- 172.21.15.26:0/4010103808 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fea58059610 con 0x7fea600b1390 2024-04-26T21:09:36.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.528+0000 7fea64a00640 1 -- 172.21.15.26:0/4010103808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea6015c0d0 con 0x7fea600b1390 2024-04-26T21:09:36.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.529+0000 7fea57400640 1 -- 172.21.15.26:0/4010103808 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16806 (secure 0 0 0) 0x7fea58058f10 con 0x7fea600b1390 2024-04-26T21:09:36.530 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:36.531 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:09:24.561115+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:08:35.481169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:08:36.609498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:09:36.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.532+0000 7fea64a00640 1 -- 172.21.15.26:0/4010103808 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fea3c06c940 msgr2=0x7fea3c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.532+0000 7fea64a00640 1 --2- 172.21.15.26:0/4010103808 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fea3c06c940 0x7fea3c06ee00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fea5000a990 tx=0x7fea50008040 comp rx=0 tx=0).stop 2024-04-26T21:09:36.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.532+0000 7fea64a00640 1 -- 172.21.15.26:0/4010103808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea600b1390 msgr2=0x7fea60158980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.532+0000 7fea64a00640 1 --2- 172.21.15.26:0/4010103808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea600b1390 0x7fea60158980 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fea5800c700 tx=0x7fea5800cbc0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.532+0000 7fea64a00640 1 -- 172.21.15.26:0/4010103808 shutdown_connections 2024-04-26T21:09:36.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.532+0000 7fea64a00640 1 --2- 172.21.15.26:0/4010103808 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fea3c06c940 0x7fea3c06ee00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.532+0000 7fea64a00640 1 --2- 172.21.15.26:0/4010103808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea600b1390 0x7fea60158980 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.532+0000 7fea64a00640 1 --2- 172.21.15.26:0/4010103808 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fea6015ffe0 0x7fea600b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.532+0000 7fea64a00640 1 --2- 172.21.15.26:0/4010103808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea60073470 0x7fea6015b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.532+0000 7fea64a00640 1 -- 172.21.15.26:0/4010103808 >> 172.21.15.26:0/4010103808 conn(0x7fea6006c690 msgr2=0x7fea6006ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:36.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.533+0000 7fea64a00640 1 -- 172.21.15.26:0/4010103808 shutdown_connections 2024-04-26T21:09:36.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.533+0000 7fea64a00640 1 -- 172.21.15.26:0/4010103808 wait complete. 2024-04-26T21:09:36.554 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:09:36.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.770+0000 7f6bc4a00640 1 Processor -- start 2024-04-26T21:09:36.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.770+0000 7f6bc4a00640 1 -- start start 2024-04-26T21:09:36.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.771+0000 7f6bc4a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bc00d1840 0x7f6bc0071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:36.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.771+0000 7f6bc4a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bc00cf690 con 0x7f6bc00d06f0 2024-04-26T21:09:36.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.771+0000 7f6bc4a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bc00cf930 con 0x7f6bc0145c20 2024-04-26T21:09:36.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.771+0000 7f6bc4a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6bc00ced10 con 0x7f6bc00d1840 2024-04-26T21:09:36.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.772+0000 7f6bb7e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6bc00d06f0 0x7f6bc00d1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60870/0 (socket says 172.21.15.26:60870) 2024-04-26T21:09:36.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.772+0000 7f6bb7e00640 1 -- 172.21.15.26:0/2836086001 learned_addr learned my addr 172.21.15.26:0/2836086001 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:36.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.772+0000 7f6bbfe00640 1 --2- 172.21.15.26:0/2836086001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bc00d1840 0x7f6bc0071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:36.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.772+0000 7f6bbea00640 1 -- 172.21.15.26:0/2836086001 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4294840864 0 0) 0x7f6bc00cf690 con 0x7f6bc00d06f0 2024-04-26T21:09:36.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.772+0000 7f6bbea00640 1 -- 172.21.15.26:0/2836086001 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ba4003520 con 0x7f6bc00d06f0 2024-04-26T21:09:36.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.772+0000 7f6bbea00640 1 -- 172.21.15.26:0/2836086001 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1593969168 0 0) 0x7f6bc00cf930 con 0x7f6bc0145c20 2024-04-26T21:09:36.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.772+0000 7f6bbea00640 1 -- 172.21.15.26:0/2836086001 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6bc00cf690 con 0x7f6bc0145c20 2024-04-26T21:09:36.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.773+0000 7f6bbea00640 1 -- 172.21.15.26:0/2836086001 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3934268643 0 0) 0x7f6ba4003520 con 0x7f6bc00d06f0 2024-04-26T21:09:36.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.773+0000 7f6bbea00640 1 -- 172.21.15.26:0/2836086001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bc00d1840 msgr2=0x7f6bc0071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:36.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.773+0000 7f6bbea00640 1 --2- 172.21.15.26:0/2836086001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bc00d1840 0x7f6bc0071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.773+0000 7f6bbea00640 1 -- 172.21.15.26:0/2836086001 >> v1:172.21.15.86:6789/0 conn(0x7f6bc0145c20 legacy=0x7f6bc0146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.773+0000 7f6bbea00640 1 -- 172.21.15.26:0/2836086001 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc00cfc10 con 0x7f6bc00d06f0 2024-04-26T21:09:36.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.773+0000 7f6bbea00640 1 -- 172.21.15.26:0/2836086001 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f6ba8001670 con 0x7f6bc00d06f0 2024-04-26T21:09:36.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.773+0000 7f6bbea00640 1 -- 172.21.15.26:0/2836086001 >> v1:172.21.15.26:6789/0 conn(0x7f6bc00d06f0 legacy=0x7f6bc00d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.773+0000 7f6bbea00640 1 --2- 172.21.15.26:0/2836086001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bc00d1840 0x7f6ba4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:36.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.773+0000 7f6bbea00640 1 --2- 172.21.15.26:0/2836086001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba40040d0 0x7f6ba4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:36.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.773+0000 7f6bbea00640 1 --2- 172.21.15.26:0/2836086001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba4004d40 0x7f6ba40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:36.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.773+0000 7f6bbea00640 1 -- 172.21.15.26:0/2836086001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba400ba70 con 0x7f6ba4004d40 2024-04-26T21:09:36.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.773+0000 7f6bbea00640 1 -- 172.21.15.26:0/2836086001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba400bc20 con 0x7f6ba40040d0 2024-04-26T21:09:36.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.773+0000 7f6bbea00640 1 -- 172.21.15.26:0/2836086001 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba400be20 con 0x7f6bc00d1840 2024-04-26T21:09:36.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.773+0000 7f6bbfe00640 1 --2- 172.21.15.26:0/2836086001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba4004d40 0x7f6ba40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:36.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.773+0000 7f6bbf400640 1 --2- 172.21.15.26:0/2836086001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bc00d1840 0x7f6ba4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:36.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.773+0000 7f6bb7e00640 1 --2- 172.21.15.26:0/2836086001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba40040d0 0x7f6ba4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:36.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.773+0000 7f6bbfe00640 1 -- 172.21.15.26:0/2836086001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bc00d1840 msgr2=0x7f6ba4003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:36.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.773+0000 7f6bbfe00640 1 --2- 172.21.15.26:0/2836086001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bc00d1840 0x7f6ba4003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.773+0000 7f6bbfe00640 1 -- 172.21.15.26:0/2836086001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba40040d0 msgr2=0x7f6ba4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:36.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.773+0000 7f6bbfe00640 1 --2- 172.21.15.26:0/2836086001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba40040d0 0x7f6ba4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.773+0000 7f6bbfe00640 1 -- 172.21.15.26:0/2836086001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba4003520 con 0x7f6ba4004d40 2024-04-26T21:09:36.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.774+0000 7f6bb7e00640 1 --2- 172.21.15.26:0/2836086001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ba40040d0 0x7f6ba4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:36.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.775+0000 7f6bbfe00640 1 --2- 172.21.15.26:0/2836086001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba4004d40 0x7f6ba40051c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8009fc0 tx=0x7f6ba4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef642ba77551ad2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:36.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.775+0000 7f6bbea00640 1 -- 172.21.15.26:0/2836086001 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8009c10 con 0x7f6ba4004d40 2024-04-26T21:09:36.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.775+0000 7f6bbea00640 1 -- 172.21.15.26:0/2836086001 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bb8036070 con 0x7f6ba4004d40 2024-04-26T21:09:36.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.775+0000 7f6bbea00640 1 -- 172.21.15.26:0/2836086001 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8031880 con 0x7f6ba4004d40 2024-04-26T21:09:36.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.775+0000 7f6bc4a00640 1 -- 172.21.15.26:0/2836086001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba4004d40 msgr2=0x7f6ba40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.775+0000 7f6bc4a00640 1 --2- 172.21.15.26:0/2836086001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ba4004d40 0x7f6ba40051c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8009fc0 tx=0x7f6ba4005830 comp rx=0 tx=0).stop 2024-04-26T21:09:36.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.775+0000 7f6bbfe00640 1 -- 172.21.15.26:0/2836086001 reap_dead start 2024-04-26T21:09:36.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.776+0000 7f6bc4a00640 1 -- 172.21.15.26:0/2836086001 shutdown_connections 2024-04-26T21:09:36.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.776+0000 7f6bc4a00640 1 -- 172.21.15.26:0/2836086001 >> 172.21.15.26:0/2836086001 conn(0x7f6bc006c690 msgr2=0x7f6bc006caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:36.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.776+0000 7f6bc4a00640 1 -- 172.21.15.26:0/2836086001 shutdown_connections 2024-04-26T21:09:36.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.776+0000 7f6bc4a00640 1 -- 172.21.15.26:0/2836086001 wait complete. 2024-04-26T21:09:36.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.777+0000 7f6bc4a00640 1 Processor -- start 2024-04-26T21:09:36.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.778+0000 7f6bc4a00640 1 -- start start 2024-04-26T21:09:36.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.778+0000 7f6bc4a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6ba4004d40 0x7f6bc015c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.778+0000 7f6bc4a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc015cc00 0x7f6bc0134090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.778+0000 7f6bc4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc01345d0 0x7f6bc015e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.778+0000 7f6bc4a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc00d01d0 con 0x7f6bc015cc00 2024-04-26T21:09:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.778+0000 7f6bc4a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc0146b60 con 0x7f6bc01345d0 2024-04-26T21:09:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.778+0000 7f6bc4a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6bc0146000 con 0x7f6ba4004d40 2024-04-26T21:09:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.779+0000 7f6bb7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc015cc00 0x7f6bc0134090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.779+0000 7f6bb7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc015cc00 0x7f6bc0134090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48936/0 (socket says 172.21.15.26:48936) 2024-04-26T21:09:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.779+0000 7f6bb7e00640 1 -- 172.21.15.26:0/481784733 learned_addr learned my addr 172.21.15.26:0/481784733 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:36.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.779+0000 7f6bbfe00640 1 --2- 172.21.15.26:0/481784733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc01345d0 0x7f6bc015e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:36.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.779+0000 7f6bbf400640 1 --2- 172.21.15.26:0/481784733 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6ba4004d40 0x7f6bc015c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:36.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.779+0000 7f6bb7e00640 1 -- 172.21.15.26:0/481784733 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6ba4004d40 msgr2=0x7f6bc015c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.779+0000 7f6bb7e00640 1 --2- 172.21.15.26:0/481784733 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6ba4004d40 0x7f6bc015c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.779+0000 7f6bb7e00640 1 -- 172.21.15.26:0/481784733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc01345d0 msgr2=0x7f6bc015e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:36.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.779+0000 7f6bb7e00640 1 --2- 172.21.15.26:0/481784733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc01345d0 0x7f6bc015e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:36.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.779+0000 7f6bb7e00640 1 -- 172.21.15.26:0/481784733 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc015d110 con 0x7f6bc015cc00 2024-04-26T21:09:36.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.779+0000 7f6bbf400640 1 --2- 172.21.15.26:0/481784733 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6ba4004d40 0x7f6bc015c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:36.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.780+0000 7f6bb7e00640 1 --2- 172.21.15.26:0/481784733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc015cc00 0x7f6bc0134090 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800d8e0 tx=0x7f6ba800dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:36.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.780+0000 7f6bb7400640 1 -- 172.21.15.26:0/481784733 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8019070 con 0x7f6bc015cc00 2024-04-26T21:09:36.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.780+0000 7f6bb7400640 1 -- 172.21.15.26:0/481784733 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ba80074e0 con 0x7f6bc015cc00 2024-04-26T21:09:36.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.780+0000 7f6bb7400640 1 -- 172.21.15.26:0/481784733 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba80049e0 con 0x7f6bc015cc00 2024-04-26T21:09:36.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.780+0000 7f6bc4a00640 1 -- 172.21.15.26:0/481784733 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc015d3f0 con 0x7f6bc015cc00 2024-04-26T21:09:36.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.780+0000 7f6bc4a00640 1 -- 172.21.15.26:0/481784733 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc015d9b0 con 0x7f6bc015cc00 2024-04-26T21:09:36.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.781+0000 7f6bc4a00640 1 -- 172.21.15.26:0/481784733 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bc006a700 con 0x7f6bc015cc00 2024-04-26T21:09:36.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.782+0000 7f6bb7400640 1 -- 172.21.15.26:0/481784733 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ba801c070 con 0x7f6bc015cc00 2024-04-26T21:09:36.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.783+0000 7f6bb7400640 1 --2- 172.21.15.26:0/481784733 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6b9c06c8f0 0x7f6b9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:36.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.783+0000 7f6bb7400640 1 -- 172.21.15.26:0/481784733 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f6ba808b670 con 0x7f6bc015cc00 2024-04-26T21:09:36.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.783+0000 7f6bbf400640 1 --2- 172.21.15.26:0/481784733 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6b9c06c8f0 0x7f6b9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:36.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.784+0000 7f6bbf400640 1 --2- 172.21.15.26:0/481784733 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6b9c06c8f0 0x7f6b9c06edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0007ae0 tx=0x7f6bb0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:36.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:36.787+0000 7f6bb7400640 1 -- 172.21.15.26:0/481784733 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6ba8059b70 con 0x7f6bc015cc00 2024-04-26T21:09:36.864 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:09:36.872 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:09:37.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.037+0000 7f6bc4a00640 1 -- 172.21.15.26:0/481784733 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6bc0070150 con 0x7f6bc015cc00 2024-04-26T21:09:37.038 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-04-26T21:09:37.037+0000 7f322fa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:09:37.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.041+0000 7f6bb7400640 1 -- 172.21.15.26:0/481784733 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f6ba8059470 con 0x7f6bc015cc00 2024-04-26T21:09:37.042 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:09:37.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.044+0000 7f6bc4a00640 1 -- 172.21.15.26:0/481784733 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6b9c06c8f0 msgr2=0x7f6b9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:37.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.044+0000 7f6bc4a00640 1 --2- 172.21.15.26:0/481784733 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6b9c06c8f0 0x7f6b9c06edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0007ae0 tx=0x7f6bb0009040 comp rx=0 tx=0).stop 2024-04-26T21:09:37.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.044+0000 7f6bc4a00640 1 -- 172.21.15.26:0/481784733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc015cc00 msgr2=0x7f6bc0134090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:37.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.044+0000 7f6bc4a00640 1 --2- 172.21.15.26:0/481784733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc015cc00 0x7f6bc0134090 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800d8e0 tx=0x7f6ba800dda0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.044+0000 7f6bc4a00640 1 -- 172.21.15.26:0/481784733 shutdown_connections 2024-04-26T21:09:37.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.044+0000 7f6bc4a00640 1 --2- 172.21.15.26:0/481784733 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f6b9c06c8f0 0x7f6b9c06edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.044+0000 7f6bc4a00640 1 --2- 172.21.15.26:0/481784733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc01345d0 0x7f6bc015e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.044+0000 7f6bc4a00640 1 --2- 172.21.15.26:0/481784733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc015cc00 0x7f6bc0134090 secure :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800d8e0 tx=0x7f6ba800dda0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.044+0000 7f6bc4a00640 1 --2- 172.21.15.26:0/481784733 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6ba4004d40 0x7f6bc015c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.044+0000 7f6bc4a00640 1 -- 172.21.15.26:0/481784733 >> 172.21.15.26:0/481784733 conn(0x7f6bc006c690 msgr2=0x7f6bc00ae850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:37.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.045+0000 7f6bc4a00640 1 -- 172.21.15.26:0/481784733 shutdown_connections 2024-04-26T21:09:37.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.045+0000 7f6bc4a00640 1 -- 172.21.15.26:0/481784733 wait complete. 2024-04-26T21:09:37.066 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:09:37.082 INFO:tasks.ceph.mds.d.smithi086.stderr: -6730> 2024-04-26T21:08:41.400+0000 7f15da600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:09:37.167 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2024-04-26T21:09:37.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.275+0000 7f3f96400640 1 Processor -- start 2024-04-26T21:09:37.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.275+0000 7f3f96400640 1 -- start start 2024-04-26T21:09:37.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.276+0000 7f3f96400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3f90160f00 0x7f3f900d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:37.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.276+0000 7f3f96400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f900d1770 con 0x7f3f900d1360 2024-04-26T21:09:37.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.276+0000 7f3f96400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f901614e0 con 0x7f3f9015d2a0 2024-04-26T21:09:37.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.276+0000 7f3f96400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3f901617c0 con 0x7f3f90160f00 2024-04-26T21:09:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.277+0000 7f3f8fe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3f900d1360 0x7f3f900cf860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60874/0 (socket says 172.21.15.26:60874) 2024-04-26T21:09:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.277+0000 7f3f8fe00640 1 -- 172.21.15.26:0/2786328493 learned_addr learned my addr 172.21.15.26:0/2786328493 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.277+0000 7f3f8ea00640 1 --2- 172.21.15.26:0/2786328493 >> v2:172.21.15.86:3300/0 conn(0x7f3f90160f00 0x7f3f900d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.277+0000 7f3f8e000640 1 -- 172.21.15.26:0/2786328493 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2481553040 0 0) 0x7f3f88001490 con 0x7f3f900d1360 2024-04-26T21:09:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.277+0000 7f3f8e000640 1 -- 172.21.15.26:0/2786328493 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f74003520 con 0x7f3f900d1360 2024-04-26T21:09:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.277+0000 7f3f8e000640 1 -- 172.21.15.26:0/2786328493 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 372032734 0 0) 0x7f3f74003520 con 0x7f3f900d1360 2024-04-26T21:09:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.277+0000 7f3f8e000640 1 -- 172.21.15.26:0/2786328493 >> v2:172.21.15.86:3300/0 conn(0x7f3f90160f00 msgr2=0x7f3f900d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.277+0000 7f3f8e000640 1 --2- 172.21.15.26:0/2786328493 >> v2:172.21.15.86:3300/0 conn(0x7f3f90160f00 0x7f3f900d0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.277+0000 7f3f8e000640 1 -- 172.21.15.26:0/2786328493 >> v1:172.21.15.86:6789/0 conn(0x7f3f9015d2a0 legacy=0x7f3f900cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.277+0000 7f3f8e000640 1 -- 172.21.15.26:0/2786328493 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f90161aa0 con 0x7f3f900d1360 2024-04-26T21:09:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.277+0000 7f3f8e000640 1 -- 172.21.15.26:0/2786328493 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f3f88001bc0 con 0x7f3f900d1360 2024-04-26T21:09:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.277+0000 7f3f8e000640 1 -- 172.21.15.26:0/2786328493 >> v1:172.21.15.26:6789/0 conn(0x7f3f900d1360 legacy=0x7f3f900cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.278+0000 7f3f8e000640 1 --2- 172.21.15.26:0/2786328493 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f740039b0 0x7f3f74003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.278+0000 7f3f8e000640 1 --2- 172.21.15.26:0/2786328493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f74004800 0x7f3f74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.278+0000 7f3f8e000640 1 --2- 172.21.15.26:0/2786328493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f740051c0 0x7f3f7400fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.278+0000 7f3f8e000640 1 -- 172.21.15.26:0/2786328493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3f901617c0 con 0x7f3f74004800 2024-04-26T21:09:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.278+0000 7f3f8e000640 1 -- 172.21.15.26:0/2786328493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3f74010450 con 0x7f3f740051c0 2024-04-26T21:09:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.278+0000 7f3f8e000640 1 -- 172.21.15.26:0/2786328493 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3f74010650 con 0x7f3f740039b0 2024-04-26T21:09:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.278+0000 7f3f8ea00640 1 --2- 172.21.15.26:0/2786328493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f74004800 0x7f3f74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.278+0000 7f3f8f400640 1 --2- 172.21.15.26:0/2786328493 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f740039b0 0x7f3f74003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.278+0000 7f3f8fe00640 1 --2- 172.21.15.26:0/2786328493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f740051c0 0x7f3f7400fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:37.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.278+0000 7f3f8ea00640 1 -- 172.21.15.26:0/2786328493 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f740039b0 msgr2=0x7f3f74003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:37.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.278+0000 7f3f8ea00640 1 --2- 172.21.15.26:0/2786328493 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f740039b0 0x7f3f74003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.278+0000 7f3f8ea00640 1 -- 172.21.15.26:0/2786328493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f740051c0 msgr2=0x7f3f7400fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:37.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.278+0000 7f3f8ea00640 1 --2- 172.21.15.26:0/2786328493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f740051c0 0x7f3f7400fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.278+0000 7f3f8ea00640 1 -- 172.21.15.26:0/2786328493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f74003520 con 0x7f3f74004800 2024-04-26T21:09:37.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.278+0000 7f3f8fe00640 1 --2- 172.21.15.26:0/2786328493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f740051c0 0x7f3f7400fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:37.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.279+0000 7f3f8fe00640 1 -- 172.21.15.26:0/2786328493 reap_dead start 2024-04-26T21:09:37.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.280+0000 7f3f8ea00640 1 --2- 172.21.15.26:0/2786328493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f74004800 0x7f3f74004c80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009ac0 tx=0x7f3f74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e76404bb1168f67 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:37.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.280+0000 7f3f8e000640 1 -- 172.21.15.26:0/2786328493 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78036070 con 0x7f3f74004800 2024-04-26T21:09:37.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.280+0000 7f3f8e000640 1 -- 172.21.15.26:0/2786328493 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f78031400 con 0x7f3f74004800 2024-04-26T21:09:37.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.280+0000 7f3f8e000640 1 -- 172.21.15.26:0/2786328493 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78009690 con 0x7f3f74004800 2024-04-26T21:09:37.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.280+0000 7f3f96400640 1 -- 172.21.15.26:0/2786328493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f74004800 msgr2=0x7f3f74004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:37.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.280+0000 7f3f96400640 1 --2- 172.21.15.26:0/2786328493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f74004800 0x7f3f74004c80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009ac0 tx=0x7f3f74005830 comp rx=0 tx=0).stop 2024-04-26T21:09:37.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.281+0000 7f3f96400640 1 -- 172.21.15.26:0/2786328493 shutdown_connections 2024-04-26T21:09:37.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.281+0000 7f3f96400640 1 --2- 172.21.15.26:0/2786328493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f74004800 0x7f3f74004c80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.281+0000 7f3f96400640 1 -- 172.21.15.26:0/2786328493 >> 172.21.15.26:0/2786328493 conn(0x7f3f90071da0 msgr2=0x7f3f900721b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:37.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.281+0000 7f3f96400640 1 -- 172.21.15.26:0/2786328493 shutdown_connections 2024-04-26T21:09:37.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.281+0000 7f3f96400640 1 -- 172.21.15.26:0/2786328493 wait complete. 2024-04-26T21:09:37.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.282+0000 7f3f96400640 1 Processor -- start 2024-04-26T21:09:37.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.283+0000 7f3f96400640 1 -- start start 2024-04-26T21:09:37.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.283+0000 7f3f96400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f74004800 0x7f3f9014e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:37.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.284+0000 7f3f96400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f9014df20 0x7f3f9006e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:37.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.284+0000 7f3f96400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f9014e3a0 0x7f3f900b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:37.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.284+0000 7f3f96400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9015d680 con 0x7f3f74004800 2024-04-26T21:09:37.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.284+0000 7f3f96400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3f90161d80 con 0x7f3f9014df20 2024-04-26T21:09:37.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.284+0000 7f3f96400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3f9015d800 con 0x7f3f9014e3a0 2024-04-26T21:09:37.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.284+0000 7f3f8f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f74004800 0x7f3f9014e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:37.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.284+0000 7f3f8f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f74004800 0x7f3f9014e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48944/0 (socket says 172.21.15.26:48944) 2024-04-26T21:09:37.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.284+0000 7f3f8f400640 1 -- 172.21.15.26:0/3017706586 learned_addr learned my addr 172.21.15.26:0/3017706586 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.284+0000 7f3f8fe00640 1 --2- 172.21.15.26:0/3017706586 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f9014e3a0 0x7f3f900b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.284+0000 7f3f8ea00640 1 --2- 172.21.15.26:0/3017706586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f9014df20 0x7f3f9006e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.284+0000 7f3f8f400640 1 -- 172.21.15.26:0/3017706586 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f9014e3a0 msgr2=0x7f3f900b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.284+0000 7f3f8f400640 1 --2- 172.21.15.26:0/3017706586 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f9014e3a0 0x7f3f900b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.284+0000 7f3f8f400640 1 -- 172.21.15.26:0/3017706586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f9014df20 msgr2=0x7f3f9006e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.284+0000 7f3f8f400640 1 --2- 172.21.15.26:0/3017706586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f9014df20 0x7f3f9006e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.284+0000 7f3f8f400640 1 -- 172.21.15.26:0/3017706586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f9015a6a0 con 0x7f3f74004800 2024-04-26T21:09:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.284+0000 7f3f8fe00640 1 --2- 172.21.15.26:0/3017706586 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f9014e3a0 0x7f3f900b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.284+0000 7f3f8ea00640 1 --2- 172.21.15.26:0/3017706586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f9014df20 0x7f3f9006e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.285+0000 7f3f8f400640 1 --2- 172.21.15.26:0/3017706586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f74004800 0x7f3f9014e7d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000e4e0 tx=0x7f3f8000e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.285+0000 7f3f87400640 1 -- 172.21.15.26:0/3017706586 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f800135f0 con 0x7f3f74004800 2024-04-26T21:09:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.285+0000 7f3f87400640 1 -- 172.21.15.26:0/3017706586 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f8001a070 con 0x7f3f74004800 2024-04-26T21:09:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.285+0000 7f3f87400640 1 -- 172.21.15.26:0/3017706586 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f800139f0 con 0x7f3f74004800 2024-04-26T21:09:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.285+0000 7f3f96400640 1 -- 172.21.15.26:0/3017706586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f9015a980 con 0x7f3f74004800 2024-04-26T21:09:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.285+0000 7f3f96400640 1 -- 172.21.15.26:0/3017706586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f9015af40 con 0x7f3f74004800 2024-04-26T21:09:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.287+0000 7f3f96400640 1 -- 172.21.15.26:0/3017706586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f900ce1b0 con 0x7f3f74004800 2024-04-26T21:09:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.287+0000 7f3f87400640 1 -- 172.21.15.26:0/3017706586 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f80013b90 con 0x7f3f74004800 2024-04-26T21:09:37.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.289+0000 7f3f87400640 1 --2- 172.21.15.26:0/3017706586 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3f6c06c8f0 0x7f3f6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:37.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.289+0000 7f3f87400640 1 -- 172.21.15.26:0/3017706586 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5911+0+0 (secure 0 0 0) 0x7f3f8008bfb0 con 0x7f3f74004800 2024-04-26T21:09:37.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.289+0000 7f3f8ea00640 1 --2- 172.21.15.26:0/3017706586 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3f6c06c8f0 0x7f3f6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:37.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.290+0000 7f3f8ea00640 1 --2- 172.21.15.26:0/3017706586 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3f6c06c8f0 0x7f3f6c06edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3f78033000 tx=0x7f3f78008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:37.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.293+0000 7f3f87400640 1 -- 172.21.15.26:0/3017706586 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f900ce1b0 con 0x7f3f74004800 2024-04-26T21:09:37.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:37.542+0000 7f3f96400640 1 -- 172.21.15.26:0/3017706586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f3f90158c10 con 0x7f3f74004800 2024-04-26T21:09:38.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.051+0000 7f3f87400640 1 -- 172.21.15.26:0/3017706586 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f3f90158c10 con 0x7f3f74004800 2024-04-26T21:09:38.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.055+0000 7f3f96400640 1 -- 172.21.15.26:0/3017706586 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3f6c06c8f0 msgr2=0x7f3f6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:38.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.055+0000 7f3f96400640 1 --2- 172.21.15.26:0/3017706586 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3f6c06c8f0 0x7f3f6c06edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3f78033000 tx=0x7f3f78008030 comp rx=0 tx=0).stop 2024-04-26T21:09:38.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.055+0000 7f3f96400640 1 -- 172.21.15.26:0/3017706586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f74004800 msgr2=0x7f3f9014e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:38.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.055+0000 7f3f96400640 1 --2- 172.21.15.26:0/3017706586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f74004800 0x7f3f9014e7d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000e4e0 tx=0x7f3f8000e9a0 comp rx=0 tx=0).stop 2024-04-26T21:09:38.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.055+0000 7f3f96400640 1 -- 172.21.15.26:0/3017706586 shutdown_connections 2024-04-26T21:09:38.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.055+0000 7f3f96400640 1 --2- 172.21.15.26:0/3017706586 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3f6c06c8f0 0x7f3f6c06edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:38.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.055+0000 7f3f96400640 1 --2- 172.21.15.26:0/3017706586 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f9014e3a0 0x7f3f900b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:38.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.055+0000 7f3f96400640 1 --2- 172.21.15.26:0/3017706586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f9014df20 0x7f3f9006e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:38.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.055+0000 7f3f96400640 1 --2- 172.21.15.26:0/3017706586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f74004800 0x7f3f9014e7d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:38.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.055+0000 7f3f96400640 1 -- 172.21.15.26:0/3017706586 >> 172.21.15.26:0/3017706586 conn(0x7f3f90071da0 msgr2=0x7f3f90159660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:38.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.055+0000 7f3f96400640 1 -- 172.21.15.26:0/3017706586 shutdown_connections 2024-04-26T21:09:38.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.056+0000 7f3f96400640 1 -- 172.21.15.26:0/3017706586 wait complete. 2024-04-26T21:09:38.081 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:09:38.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.299+0000 7f658fa00640 1 Processor -- start 2024-04-26T21:09:38.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.299+0000 7f658fa00640 1 -- start start 2024-04-26T21:09:38.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.301+0000 7f658fa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f65880b3440 0x7f6588143270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:38.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.301+0000 7f658fa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6588140210 0x7f658815dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:38.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.301+0000 7f658fa00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65881611c0 con 0x7f658813fe30 2024-04-26T21:09:38.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.301+0000 7f658fa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f65881437b0 con 0x7f65880b3440 2024-04-26T21:09:38.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.301+0000 7f658fa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f65881614a0 con 0x7f6588140210 2024-04-26T21:09:38.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.301+0000 7f658d000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f658813fe30 0x7f65880b2d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60888/0 (socket says 172.21.15.26:60888) 2024-04-26T21:09:38.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.301+0000 7f658d000640 1 -- 172.21.15.26:0/2132166272 learned_addr learned my addr 172.21.15.26:0/2132166272 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:38.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.301+0000 7f6587400640 1 -- 172.21.15.26:0/2132166272 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3043798009 0 0) 0x7f65881611c0 con 0x7f658813fe30 2024-04-26T21:09:38.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.302+0000 7f6587400640 1 -- 172.21.15.26:0/2132166272 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f656c003520 con 0x7f658813fe30 2024-04-26T21:09:38.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.301+0000 7f6587e00640 1 --2- 172.21.15.26:0/2132166272 >> v2:172.21.15.86:3300/0 conn(0x7f65880b3440 0x7f6588143270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:38.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.302+0000 7f658da00640 1 --2- 172.21.15.26:0/2132166272 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6588140210 0x7f658815dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:38.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.302+0000 7f6587400640 1 -- 172.21.15.26:0/2132166272 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1761104538 0 0) 0x7f656c003520 con 0x7f658813fe30 2024-04-26T21:09:38.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.302+0000 7f6587400640 1 -- 172.21.15.26:0/2132166272 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6588140210 msgr2=0x7f658815dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:38.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.302+0000 7f6587400640 1 --2- 172.21.15.26:0/2132166272 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6588140210 0x7f658815dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:38.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.302+0000 7f6587400640 1 -- 172.21.15.26:0/2132166272 >> v2:172.21.15.86:3300/0 conn(0x7f65880b3440 msgr2=0x7f6588143270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:38.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.302+0000 7f6587400640 1 --2- 172.21.15.26:0/2132166272 >> v2:172.21.15.86:3300/0 conn(0x7f65880b3440 0x7f6588143270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:38.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.302+0000 7f6587400640 1 -- 172.21.15.26:0/2132166272 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6588161780 con 0x7f658813fe30 2024-04-26T21:09:38.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.302+0000 7f6587400640 1 -- 172.21.15.26:0/2132166272 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f6578001690 con 0x7f658813fe30 2024-04-26T21:09:38.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.302+0000 7f6587400640 1 -- 172.21.15.26:0/2132166272 >> v1:172.21.15.26:6789/0 conn(0x7f658813fe30 legacy=0x7f65880b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:38.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.302+0000 7f6587400640 1 --2- 172.21.15.26:0/2132166272 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6588140210 0x7f656c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:38.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.302+0000 7f6587400640 1 --2- 172.21.15.26:0/2132166272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c003f30 0x7f656c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:38.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.302+0000 7f6587400640 1 --2- 172.21.15.26:0/2132166272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f656c004d40 0x7f656c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:38.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.302+0000 7f6587400640 1 -- 172.21.15.26:0/2132166272 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f65881437b0 con 0x7f656c003f30 2024-04-26T21:09:38.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.302+0000 7f6587400640 1 -- 172.21.15.26:0/2132166272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65881614a0 con 0x7f656c004d40 2024-04-26T21:09:38.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.302+0000 7f6587400640 1 -- 172.21.15.26:0/2132166272 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f656c00ba10 con 0x7f6588140210 2024-04-26T21:09:38.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.302+0000 7f6587e00640 1 --2- 172.21.15.26:0/2132166272 >> v2:172.21.15.86:3300/0 conn(0x7f65880b3440 0x7f6588143270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:09:38.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.303+0000 7f6587e00640 1 --2- 172.21.15.26:0/2132166272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c003f30 0x7f656c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:38.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.303+0000 7f658d000640 1 --2- 172.21.15.26:0/2132166272 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6588140210 0x7f656c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:38.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.303+0000 7f658da00640 1 --2- 172.21.15.26:0/2132166272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f656c004d40 0x7f656c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:38.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.303+0000 7f6587e00640 1 -- 172.21.15.26:0/2132166272 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6588140210 msgr2=0x7f656c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:38.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.303+0000 7f6587e00640 1 --2- 172.21.15.26:0/2132166272 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6588140210 0x7f656c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:38.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.303+0000 7f6587e00640 1 -- 172.21.15.26:0/2132166272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f656c004d40 msgr2=0x7f656c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:38.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.303+0000 7f6587e00640 1 --2- 172.21.15.26:0/2132166272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f656c004d40 0x7f656c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:38.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.303+0000 7f6587e00640 1 -- 172.21.15.26:0/2132166272 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f656c003520 con 0x7f656c003f30 2024-04-26T21:09:38.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.304+0000 7f6587e00640 1 --2- 172.21.15.26:0/2132166272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c003f30 0x7f656c004800 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f65740098a0 tx=0x7f656c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798a0256c118f710 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:38.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.304+0000 7f6587400640 1 -- 172.21.15.26:0/2132166272 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574037070 con 0x7f656c003f30 2024-04-26T21:09:38.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.304+0000 7f6587400640 1 -- 172.21.15.26:0/2132166272 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f657402ad20 con 0x7f656c003f30 2024-04-26T21:09:38.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.305+0000 7f6587400640 1 -- 172.21.15.26:0/2132166272 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657400f040 con 0x7f656c003f30 2024-04-26T21:09:38.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.305+0000 7f658fa00640 1 -- 172.21.15.26:0/2132166272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c003f30 msgr2=0x7f656c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:38.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.305+0000 7f658fa00640 1 --2- 172.21.15.26:0/2132166272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c003f30 0x7f656c004800 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f65740098a0 tx=0x7f656c005830 comp rx=0 tx=0).stop 2024-04-26T21:09:38.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.305+0000 7f658da00640 1 -- 172.21.15.26:0/2132166272 reap_dead start 2024-04-26T21:09:38.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.305+0000 7f658fa00640 1 -- 172.21.15.26:0/2132166272 shutdown_connections 2024-04-26T21:09:38.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.305+0000 7f658fa00640 1 -- 172.21.15.26:0/2132166272 >> 172.21.15.26:0/2132166272 conn(0x7f658806c420 msgr2=0x7f658806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:38.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.305+0000 7f658fa00640 1 -- 172.21.15.26:0/2132166272 shutdown_connections 2024-04-26T21:09:38.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.305+0000 7f658fa00640 1 -- 172.21.15.26:0/2132166272 wait complete. 2024-04-26T21:09:38.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.307+0000 7f658fa00640 1 Processor -- start 2024-04-26T21:09:38.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.307+0000 7f658fa00640 1 -- start start 2024-04-26T21:09:38.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.308+0000 7f658fa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c003f30 0x7f6588073020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:38.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.308+0000 7f658fa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65881563b0 0x7f6588156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:38.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.308+0000 7f658fa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f658813ca30 0x7f658813ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:38.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.309+0000 7f658fa00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6588160d50 con 0x7f656c003f30 2024-04-26T21:09:38.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.309+0000 7f658fa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65880cda70 con 0x7f65881563b0 2024-04-26T21:09:38.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.309+0000 7f658fa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f6588160ed0 con 0x7f658813ca30 2024-04-26T21:09:38.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.309+0000 7f658d000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c003f30 0x7f6588073020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:38.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.309+0000 7f658d000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c003f30 0x7f6588073020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48956/0 (socket says 172.21.15.26:48956) 2024-04-26T21:09:38.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.309+0000 7f658d000640 1 -- 172.21.15.26:0/2381927451 learned_addr learned my addr 172.21.15.26:0/2381927451 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:38.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.309+0000 7f6587e00640 1 --2- 172.21.15.26:0/2381927451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65881563b0 0x7f6588156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:38.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.309+0000 7f658da00640 1 --2- 172.21.15.26:0/2381927451 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f658813ca30 0x7f658813ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:38.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.309+0000 7f658d000640 1 -- 172.21.15.26:0/2381927451 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f658813ca30 msgr2=0x7f658813ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:38.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.309+0000 7f658d000640 1 --2- 172.21.15.26:0/2381927451 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f658813ca30 0x7f658813ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:38.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.309+0000 7f658d000640 1 -- 172.21.15.26:0/2381927451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65881563b0 msgr2=0x7f6588156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:38.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.309+0000 7f658d000640 1 --2- 172.21.15.26:0/2381927451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65881563b0 0x7f6588156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:38.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.309+0000 7f658d000640 1 -- 172.21.15.26:0/2381927451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658813d3f0 con 0x7f656c003f30 2024-04-26T21:09:38.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.309+0000 7f6587e00640 1 --2- 172.21.15.26:0/2381927451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65881563b0 0x7f6588156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:38.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.309+0000 7f658d000640 1 --2- 172.21.15.26:0/2381927451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c003f30 0x7f6588073020 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f657800e840 tx=0x7f657800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:38.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.310+0000 7f6584c00640 1 -- 172.21.15.26:0/2381927451 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657800c7f0 con 0x7f656c003f30 2024-04-26T21:09:38.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.310+0000 7f6584c00640 1 -- 172.21.15.26:0/2381927451 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6578018070 con 0x7f656c003f30 2024-04-26T21:09:38.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.310+0000 7f6584c00640 1 -- 172.21.15.26:0/2381927451 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657800cbb0 con 0x7f656c003f30 2024-04-26T21:09:38.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.310+0000 7f658fa00640 1 -- 172.21.15.26:0/2381927451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6588154920 con 0x7f656c003f30 2024-04-26T21:09:38.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.310+0000 7f658fa00640 1 -- 172.21.15.26:0/2381927451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6588154ee0 con 0x7f656c003f30 2024-04-26T21:09:38.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.311+0000 7f658fa00640 1 -- 172.21.15.26:0/2381927451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f658806a700 con 0x7f656c003f30 2024-04-26T21:09:38.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.312+0000 7f6584c00640 1 -- 172.21.15.26:0/2381927451 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f657801d460 con 0x7f656c003f30 2024-04-26T21:09:38.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.313+0000 7f6584c00640 1 --2- 172.21.15.26:0/2381927451 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f656806c8f0 0x7f656806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:38.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.313+0000 7f6584c00640 1 -- 172.21.15.26:0/2381927451 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5911+0+0 (secure 0 0 0) 0x7f657808bc00 con 0x7f656c003f30 2024-04-26T21:09:38.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.313+0000 7f6587e00640 1 --2- 172.21.15.26:0/2381927451 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f656806c8f0 0x7f656806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:38.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.314+0000 7f6587e00640 1 --2- 172.21.15.26:0/2381927451 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f656806c8f0 0x7f656806edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6574034000 tx=0x7f657402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:38.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.318+0000 7f6584c00640 1 -- 172.21.15.26:0/2381927451 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6578059ec0 con 0x7f656c003f30 2024-04-26T21:09:38.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:38.524+0000 7f658fa00640 1 -- 172.21.15.26:0/2381927451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f658806fe20 con 0x7f656c003f30 2024-04-26T21:09:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.054+0000 7f6584c00640 1 -- 172.21.15.26:0/2381927451 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) ==== 181+0+0 (secure 0 0 0) 0x7f65780597c0 con 0x7f656c003f30 2024-04-26T21:09:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:09:39.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.058+0000 7f658fa00640 1 -- 172.21.15.26:0/2381927451 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f656806c8f0 msgr2=0x7f656806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:39.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.058+0000 7f658fa00640 1 --2- 172.21.15.26:0/2381927451 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f656806c8f0 0x7f656806edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6574034000 tx=0x7f657402e000 comp rx=0 tx=0).stop 2024-04-26T21:09:39.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.058+0000 7f658fa00640 1 -- 172.21.15.26:0/2381927451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c003f30 msgr2=0x7f6588073020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:39.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.058+0000 7f658fa00640 1 --2- 172.21.15.26:0/2381927451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c003f30 0x7f6588073020 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f657800e840 tx=0x7f657800ed00 comp rx=0 tx=0).stop 2024-04-26T21:09:39.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.059+0000 7f658fa00640 1 -- 172.21.15.26:0/2381927451 shutdown_connections 2024-04-26T21:09:39.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.059+0000 7f658fa00640 1 --2- 172.21.15.26:0/2381927451 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f656806c8f0 0x7f656806edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:39.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.059+0000 7f658fa00640 1 --2- 172.21.15.26:0/2381927451 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f658813ca30 0x7f658813ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:39.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.059+0000 7f658fa00640 1 --2- 172.21.15.26:0/2381927451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65881563b0 0x7f6588156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:39.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.059+0000 7f658fa00640 1 --2- 172.21.15.26:0/2381927451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c003f30 0x7f6588073020 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:39.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.059+0000 7f658fa00640 1 -- 172.21.15.26:0/2381927451 >> 172.21.15.26:0/2381927451 conn(0x7f658806c420 msgr2=0x7f658813ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:39.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.059+0000 7f658fa00640 1 -- 172.21.15.26:0/2381927451 shutdown_connections 2024-04-26T21:09:39.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.059+0000 7f658fa00640 1 -- 172.21.15.26:0/2381927451 wait complete. 2024-04-26T21:09:39.087 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:09:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.296+0000 7f04c2000640 1 Processor -- start 2024-04-26T21:09:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.297+0000 7f04c2000640 1 -- start start 2024-04-26T21:09:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.298+0000 7f04c2000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f04bc0d3e10 0x7f04bc0b5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.298+0000 7f04c2000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04bc0d1010 con 0x7f04bc15f660 2024-04-26T21:09:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.298+0000 7f04c2000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04bc0d5de0 con 0x7f04bc0d2f00 2024-04-26T21:09:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.298+0000 7f04c2000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f04bc0b4780 con 0x7f04bc0d3e10 2024-04-26T21:09:39.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.298+0000 7f04bb400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f04bc15f660 0x7f04bc0d3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32844/0 (socket says 172.21.15.26:32844) 2024-04-26T21:09:39.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.298+0000 7f04bb400640 1 -- 172.21.15.26:0/1165724940 learned_addr learned my addr 172.21.15.26:0/1165724940 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.299+0000 7f04baa00640 1 --2- 172.21.15.26:0/1165724940 >> v2:172.21.15.86:3300/0 conn(0x7f04bc0d3e10 0x7f04bc0b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.299+0000 7f04ba000640 1 -- 172.21.15.26:0/1165724940 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1169080643 0 0) 0x7f04bc0d1010 con 0x7f04bc15f660 2024-04-26T21:09:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.299+0000 7f04ba000640 1 -- 172.21.15.26:0/1165724940 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f049c003520 con 0x7f04bc15f660 2024-04-26T21:09:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.299+0000 7f04ba000640 1 -- 172.21.15.26:0/1165724940 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2057694012 0 0) 0x7f049c003520 con 0x7f04bc15f660 2024-04-26T21:09:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.299+0000 7f04ba000640 1 -- 172.21.15.26:0/1165724940 >> v2:172.21.15.86:3300/0 conn(0x7f04bc0d3e10 msgr2=0x7f04bc0b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.299+0000 7f04ba000640 1 --2- 172.21.15.26:0/1165724940 >> v2:172.21.15.86:3300/0 conn(0x7f04bc0d3e10 0x7f04bc0b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.299+0000 7f04ba000640 1 -- 172.21.15.26:0/1165724940 >> v1:172.21.15.86:6789/0 conn(0x7f04bc0d2f00 legacy=0x7f04bc0b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.299+0000 7f04ba000640 1 -- 172.21.15.26:0/1165724940 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc0b4a60 con 0x7f04bc15f660 2024-04-26T21:09:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.300+0000 7f04ba000640 1 -- 172.21.15.26:0/1165724940 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f04ac001670 con 0x7f04bc15f660 2024-04-26T21:09:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.300+0000 7f04ba000640 1 -- 172.21.15.26:0/1165724940 >> v1:172.21.15.26:6789/0 conn(0x7f04bc15f660 legacy=0x7f04bc0d3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.300+0000 7f04ba000640 1 --2- 172.21.15.26:0/1165724940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f049c003a00 0x7f049c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.300+0000 7f04ba000640 1 --2- 172.21.15.26:0/1165724940 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f049c004800 0x7f049c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.300+0000 7f04ba000640 1 --2- 172.21.15.26:0/1165724940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f049c0051c0 0x7f049c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.300+0000 7f04ba000640 1 -- 172.21.15.26:0/1165724940 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc0b4780 con 0x7f049c0051c0 2024-04-26T21:09:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.300+0000 7f04ba000640 1 -- 172.21.15.26:0/1165724940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f049c00c0c0 con 0x7f049c003a00 2024-04-26T21:09:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.300+0000 7f04ba000640 1 -- 172.21.15.26:0/1165724940 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f049c00c2c0 con 0x7f049c004800 2024-04-26T21:09:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.300+0000 7f04bbe00640 1 --2- 172.21.15.26:0/1165724940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f049c0051c0 0x7f049c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.300+0000 7f04bb400640 1 --2- 172.21.15.26:0/1165724940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f049c003a00 0x7f049c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.300+0000 7f04baa00640 1 --2- 172.21.15.26:0/1165724940 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f049c004800 0x7f049c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.301+0000 7f04bbe00640 1 -- 172.21.15.26:0/1165724940 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f049c004800 msgr2=0x7f049c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.301+0000 7f04bbe00640 1 --2- 172.21.15.26:0/1165724940 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f049c004800 0x7f049c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.301+0000 7f04bbe00640 1 -- 172.21.15.26:0/1165724940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f049c003a00 msgr2=0x7f049c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.301+0000 7f04bbe00640 1 --2- 172.21.15.26:0/1165724940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f049c003a00 0x7f049c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.301+0000 7f04bbe00640 1 -- 172.21.15.26:0/1165724940 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049c003520 con 0x7f049c0051c0 2024-04-26T21:09:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.301+0000 7f04bbe00640 1 -- 172.21.15.26:0/1165724940 reap_dead start 2024-04-26T21:09:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.302+0000 7f04bbe00640 1 --2- 172.21.15.26:0/1165724940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f049c0051c0 0x7f049c00b9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f04b4001750 tx=0x7f049c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13020c078abfa5d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.302+0000 7f04ba000640 1 -- 172.21.15.26:0/1165724940 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b4001370 con 0x7f049c0051c0 2024-04-26T21:09:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.302+0000 7f04ba000640 1 -- 172.21.15.26:0/1165724940 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b4005310 con 0x7f049c0051c0 2024-04-26T21:09:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.302+0000 7f04ba000640 1 -- 172.21.15.26:0/1165724940 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b402abe0 con 0x7f049c0051c0 2024-04-26T21:09:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.302+0000 7f04c2000640 1 -- 172.21.15.26:0/1165724940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f049c0051c0 msgr2=0x7f049c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.302+0000 7f04c2000640 1 --2- 172.21.15.26:0/1165724940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f049c0051c0 0x7f049c00b9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f04b4001750 tx=0x7f049c005830 comp rx=0 tx=0).stop 2024-04-26T21:09:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.303+0000 7f04c2000640 1 -- 172.21.15.26:0/1165724940 shutdown_connections 2024-04-26T21:09:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.303+0000 7f04c2000640 1 --2- 172.21.15.26:0/1165724940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f049c0051c0 0x7f049c00b9c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.303+0000 7f04c2000640 1 -- 172.21.15.26:0/1165724940 >> 172.21.15.26:0/1165724940 conn(0x7f04bc0d5120 msgr2=0x7f04bc0d5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.303+0000 7f04c2000640 1 -- 172.21.15.26:0/1165724940 shutdown_connections 2024-04-26T21:09:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.303+0000 7f04c2000640 1 -- 172.21.15.26:0/1165724940 wait complete. 2024-04-26T21:09:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.304+0000 7f04c2000640 1 Processor -- start 2024-04-26T21:09:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.305+0000 7f04c2000640 1 -- start start 2024-04-26T21:09:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.306+0000 7f04c2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f049c0051c0 0x7f04bc1f6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.306+0000 7f04c2000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04bc0a0da0 0x7f04bc0a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.306+0000 7f04c2000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04bc09ee60 0x7f04bc09f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.306+0000 7f04c2000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc0d2370 con 0x7f04bc0a0da0 2024-04-26T21:09:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.306+0000 7f04c2000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc0d21f0 con 0x7f049c0051c0 2024-04-26T21:09:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.306+0000 7f04c2000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f04bc0d24f0 con 0x7f04bc09ee60 2024-04-26T21:09:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.306+0000 7f04baa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04bc0a0da0 0x7f04bc0a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.306+0000 7f04baa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04bc0a0da0 0x7f04bc0a1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47870/0 (socket says 172.21.15.26:47870) 2024-04-26T21:09:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.306+0000 7f04baa00640 1 -- 172.21.15.26:0/2890199131 learned_addr learned my addr 172.21.15.26:0/2890199131 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.306+0000 7f04bbe00640 1 --2- 172.21.15.26:0/2890199131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04bc09ee60 0x7f04bc09f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.306+0000 7f04bb400640 1 --2- 172.21.15.26:0/2890199131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f049c0051c0 0x7f04bc1f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.307+0000 7f04baa00640 1 -- 172.21.15.26:0/2890199131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04bc09ee60 msgr2=0x7f04bc09f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.307+0000 7f04baa00640 1 --2- 172.21.15.26:0/2890199131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04bc09ee60 0x7f04bc09f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.307+0000 7f04baa00640 1 -- 172.21.15.26:0/2890199131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f049c0051c0 msgr2=0x7f04bc1f6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.307+0000 7f04baa00640 1 --2- 172.21.15.26:0/2890199131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f049c0051c0 0x7f04bc1f6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.307+0000 7f04baa00640 1 -- 172.21.15.26:0/2890199131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc0cd3c0 con 0x7f04bc0a0da0 2024-04-26T21:09:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.307+0000 7f04bb400640 1 --2- 172.21.15.26:0/2890199131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f049c0051c0 0x7f04bc1f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:09:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.307+0000 7f04baa00640 1 --2- 172.21.15.26:0/2890199131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04bc0a0da0 0x7f04bc0a1220 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f04a4007f00 tx=0x7f04a40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.307+0000 7f04b3400640 1 -- 172.21.15.26:0/2890199131 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a401a020 con 0x7f04bc0a0da0 2024-04-26T21:09:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.307+0000 7f04b3400640 1 -- 172.21.15.26:0/2890199131 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a4016470 con 0x7f04bc0a0da0 2024-04-26T21:09:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.307+0000 7f04b3400640 1 -- 172.21.15.26:0/2890199131 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a4016860 con 0x7f04bc0a0da0 2024-04-26T21:09:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.307+0000 7f04c2000640 1 -- 172.21.15.26:0/2890199131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04bc0cd6a0 con 0x7f04bc0a0da0 2024-04-26T21:09:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.307+0000 7f04c2000640 1 -- 172.21.15.26:0/2890199131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04bc0cdc60 con 0x7f04bc0a0da0 2024-04-26T21:09:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.309+0000 7f04c2000640 1 -- 172.21.15.26:0/2890199131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04bc0693d0 con 0x7f04bc0a0da0 2024-04-26T21:09:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.310+0000 7f04b3400640 1 -- 172.21.15.26:0/2890199131 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04a4005d10 con 0x7f04bc0a0da0 2024-04-26T21:09:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.310+0000 7f04b3400640 1 --2- 172.21.15.26:0/2890199131 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f049806c920 0x7f049806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:39.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.311+0000 7f04b3400640 1 -- 172.21.15.26:0/2890199131 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5473+0+0 (secure 0 0 0) 0x7f04a408c8c0 con 0x7f04bc0a0da0 2024-04-26T21:09:39.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.311+0000 7f04bb400640 1 --2- 172.21.15.26:0/2890199131 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f049806c920 0x7f049806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:39.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.311+0000 7f04bb400640 1 --2- 172.21.15.26:0/2890199131 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f049806c920 0x7f049806ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f04ac001050 tx=0x7f04ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:39.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.315+0000 7f04b3400640 1 -- 172.21.15.26:0/2890199131 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f04a4024030 con 0x7f04bc0a0da0 2024-04-26T21:09:39.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:39.518+0000 7f04c2000640 1 -- 172.21.15.26:0/2890199131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f04bc0d34e0 con 0x7f04bc0a0da0 2024-04-26T21:09:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.065+0000 7f04b3400640 1 -- 172.21.15.26:0/2890199131 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) ==== 169+0+0 (secure 0 0 0) 0x7f04a405ad30 con 0x7f04bc0a0da0 2024-04-26T21:09:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-04-26T21:09:40.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.069+0000 7f04c2000640 1 -- 172.21.15.26:0/2890199131 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f049806c920 msgr2=0x7f049806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:40.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.069+0000 7f04c2000640 1 --2- 172.21.15.26:0/2890199131 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f049806c920 0x7f049806ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f04ac001050 tx=0x7f04ac009040 comp rx=0 tx=0).stop 2024-04-26T21:09:40.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.069+0000 7f04c2000640 1 -- 172.21.15.26:0/2890199131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04bc0a0da0 msgr2=0x7f04bc0a1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:40.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.069+0000 7f04c2000640 1 --2- 172.21.15.26:0/2890199131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04bc0a0da0 0x7f04bc0a1220 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f04a4007f00 tx=0x7f04a40115a0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.069+0000 7f04c2000640 1 -- 172.21.15.26:0/2890199131 shutdown_connections 2024-04-26T21:09:40.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.069+0000 7f04c2000640 1 --2- 172.21.15.26:0/2890199131 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f049806c920 0x7f049806ede0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.069+0000 7f04c2000640 1 --2- 172.21.15.26:0/2890199131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04bc09ee60 0x7f04bc09f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.069+0000 7f04c2000640 1 --2- 172.21.15.26:0/2890199131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04bc0a0da0 0x7f04bc0a1220 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.069+0000 7f04c2000640 1 --2- 172.21.15.26:0/2890199131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f049c0051c0 0x7f04bc1f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.069+0000 7f04c2000640 1 -- 172.21.15.26:0/2890199131 >> 172.21.15.26:0/2890199131 conn(0x7f04bc0d5120 msgr2=0x7f04bc0b5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:40.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.070+0000 7f04c2000640 1 -- 172.21.15.26:0/2890199131 shutdown_connections 2024-04-26T21:09:40.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.070+0000 7f04c2000640 1 -- 172.21.15.26:0/2890199131 wait complete. 2024-04-26T21:09:40.099 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T21:09:40.099 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T21:09:40.100 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:09:40.100 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:09:40.100 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T21:09:40.100 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T21:09:40.100 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:09:40.100 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:09:40.101 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T21:09:40.101 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T21:09:40.101 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:09:40.101 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:09:40.101 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T21:09:40.101 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T21:09:40.101 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:09:40.101 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:09:40.119 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:09:40.119 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T21:09:40.120 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:09:40.120 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T21:09:40.125 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:09:40.125 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T21:09:40.126 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:09:40.127 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T21:09:40.167 INFO:tasks.ceph.mds.a:Started 2024-04-26T21:09:40.167 DEBUG:teuthology.parallel:result is None 2024-04-26T21:09:40.170 INFO:tasks.ceph.mds.d:Started 2024-04-26T21:09:40.171 DEBUG:teuthology.parallel:result is None 2024-04-26T21:09:40.172 INFO:tasks.ceph.mds.c:Started 2024-04-26T21:09:40.172 DEBUG:teuthology.parallel:result is None 2024-04-26T21:09:40.174 INFO:tasks.ceph.mds.b:Started 2024-04-26T21:09:40.175 DEBUG:teuthology.parallel:result is None 2024-04-26T21:09:40.175 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T21:09:40.320 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-04-26T21:09:40.321 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-04-26T21:09:40.324 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2024-04-26T21:09:40.325 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2024-04-26T21:09:40.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.361+0000 7f47f2c00640 1 Processor -- start 2024-04-26T21:09:40.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.362+0000 7f47f2c00640 1 -- start start 2024-04-26T21:09:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.363+0000 7f47f2c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f47ec146990 0x7f47ec0b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.363+0000 7f47f2c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f47ec06ec20 0x7f47ec0d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.363+0000 7f47f2c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec0d1210 0x7f47ec072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.363+0000 7f47f2c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f47ec146d70 con 0x7f47ec146990 2024-04-26T21:09:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.363+0000 7f47f2c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f47ec0d1640 con 0x7f47ec06ec20 2024-04-26T21:09:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.363+0000 7f47f2c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f47ec0b4480 con 0x7f47ec0d1210 2024-04-26T21:09:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.363+0000 7f47ebe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f47ec146990 0x7f47ec0b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.363+0000 7f47e3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f47ec06ec20 0x7f47ec0d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.363+0000 7f47ebe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f47ec146990 0x7f47ec0b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47908/0 (socket says 172.21.15.26:47908) 2024-04-26T21:09:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.363+0000 7f47f0c00640 1 --2- 172.21.15.26:0/2964847740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec0d1210 0x7f47ec072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.363+0000 7f47ebe00640 1 -- 172.21.15.26:0/2964847740 learned_addr learned my addr 172.21.15.26:0/2964847740 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:40.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.364+0000 7f47e3e00640 1 -- 172.21.15.26:0/2964847740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec0d1210 msgr2=0x7f47ec072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:40.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.364+0000 7f47e3e00640 1 --2- 172.21.15.26:0/2964847740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec0d1210 0x7f47ec072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.364+0000 7f47e3e00640 1 -- 172.21.15.26:0/2964847740 >> v2:172.21.15.26:3300/0 conn(0x7f47ec146990 msgr2=0x7f47ec0b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:40.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.364+0000 7f47e3e00640 1 --2- 172.21.15.26:0/2964847740 >> v2:172.21.15.26:3300/0 conn(0x7f47ec146990 0x7f47ec0b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.364+0000 7f47e3e00640 1 -- 172.21.15.26:0/2964847740 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ec073b70 con 0x7f47ec06ec20 2024-04-26T21:09:40.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.365+0000 7f47e3e00640 1 --2- 172.21.15.26:0/2964847740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec06ec20 0x7f47ec0d0cd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f47ec15b0c0 tx=0x7f47d402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0f53402408164fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:40.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.365+0000 7f47eb400640 1 -- 172.21.15.26:0/2964847740 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d4037070 con 0x7f47ec06ec20 2024-04-26T21:09:40.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.366+0000 7f47eb400640 1 -- 172.21.15.26:0/2964847740 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47d4032440 con 0x7f47ec06ec20 2024-04-26T21:09:40.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.366+0000 7f47eb400640 1 -- 172.21.15.26:0/2964847740 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d402ae60 con 0x7f47ec06ec20 2024-04-26T21:09:40.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.366+0000 7f47f2c00640 1 -- 172.21.15.26:0/2964847740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec06ec20 msgr2=0x7f47ec0d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:40.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.366+0000 7f47f2c00640 1 --2- 172.21.15.26:0/2964847740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec06ec20 0x7f47ec0d0cd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f47ec15b0c0 tx=0x7f47d402a110 comp rx=0 tx=0).stop 2024-04-26T21:09:40.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.366+0000 7f47f2c00640 1 -- 172.21.15.26:0/2964847740 shutdown_connections 2024-04-26T21:09:40.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.366+0000 7f47f2c00640 1 --2- 172.21.15.26:0/2964847740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec0d1210 0x7f47ec072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.366+0000 7f47f2c00640 1 --2- 172.21.15.26:0/2964847740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec06ec20 0x7f47ec0d0cd0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.366+0000 7f47f2c00640 1 --2- 172.21.15.26:0/2964847740 >> v2:172.21.15.26:3300/0 conn(0x7f47ec146990 0x7f47ec0b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.366+0000 7f47f2c00640 1 -- 172.21.15.26:0/2964847740 >> 172.21.15.26:0/2964847740 conn(0x7f47ec0cd840 msgr2=0x7f47ec0cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:40.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.366+0000 7f47f2c00640 1 -- 172.21.15.26:0/2964847740 shutdown_connections 2024-04-26T21:09:40.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.366+0000 7f47f2c00640 1 -- 172.21.15.26:0/2964847740 wait complete. 2024-04-26T21:09:40.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.368+0000 7f47f2c00640 1 Processor -- start 2024-04-26T21:09:40.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.368+0000 7f47f2c00640 1 -- start start 2024-04-26T21:09:40.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.369+0000 7f47f2c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ec06ec20 0x7f47ec160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:40.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.369+0000 7f47f2c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec0d1210 0x7f47ec1f5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:40.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.369+0000 7f47f2c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec146990 0x7f47ec1f5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:40.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.369+0000 7f47f2c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec06e420 con 0x7f47ec06ec20 2024-04-26T21:09:40.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.369+0000 7f47f2c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec074130 con 0x7f47ec0d1210 2024-04-26T21:09:40.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.369+0000 7f47f2c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f47ec06e700 con 0x7f47ec146990 2024-04-26T21:09:40.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.369+0000 7f47ebe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ec06ec20 0x7f47ec160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:40.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.369+0000 7f47ebe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ec06ec20 0x7f47ec160600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47910/0 (socket says 172.21.15.26:47910) 2024-04-26T21:09:40.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.369+0000 7f47ebe00640 1 -- 172.21.15.26:0/1090570096 learned_addr learned my addr 172.21.15.26:0/1090570096 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:40.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.370+0000 7f47e3e00640 1 --2- 172.21.15.26:0/1090570096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec0d1210 0x7f47ec1f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:40.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.370+0000 7f47f0c00640 1 --2- 172.21.15.26:0/1090570096 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec146990 0x7f47ec1f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:40.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.370+0000 7f47ebe00640 1 -- 172.21.15.26:0/1090570096 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec146990 msgr2=0x7f47ec1f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:40.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.370+0000 7f47ebe00640 1 --2- 172.21.15.26:0/1090570096 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec146990 0x7f47ec1f5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.370+0000 7f47ebe00640 1 -- 172.21.15.26:0/1090570096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec0d1210 msgr2=0x7f47ec1f5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:40.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.370+0000 7f47ebe00640 1 --2- 172.21.15.26:0/1090570096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec0d1210 0x7f47ec1f5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:40.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.370+0000 7f47ebe00640 1 -- 172.21.15.26:0/1090570096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ec13adc0 con 0x7f47ec06ec20 2024-04-26T21:09:40.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.370+0000 7f47f0c00640 1 --2- 172.21.15.26:0/1090570096 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec146990 0x7f47ec1f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:09:40.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.370+0000 7f47e3e00640 1 --2- 172.21.15.26:0/1090570096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec0d1210 0x7f47ec1f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:09:40.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.370+0000 7f47ebe00640 1 --2- 172.21.15.26:0/1090570096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ec06ec20 0x7f47ec160600 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00e8b0 tx=0x7f47dc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:40.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.371+0000 7f47e8c00640 1 -- 172.21.15.26:0/1090570096 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc017070 con 0x7f47ec06ec20 2024-04-26T21:09:40.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.371+0000 7f47e8c00640 1 -- 172.21.15.26:0/1090570096 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47dc00c940 con 0x7f47ec06ec20 2024-04-26T21:09:40.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.371+0000 7f47e8c00640 1 -- 172.21.15.26:0/1090570096 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc00cd10 con 0x7f47ec06ec20 2024-04-26T21:09:40.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.371+0000 7f47f2c00640 1 -- 172.21.15.26:0/1090570096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47ec13b0a0 con 0x7f47ec06ec20 2024-04-26T21:09:40.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.371+0000 7f47f2c00640 1 -- 172.21.15.26:0/1090570096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47ec15e850 con 0x7f47ec06ec20 2024-04-26T21:09:40.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.372+0000 7f47f2c00640 1 -- 172.21.15.26:0/1090570096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47ec0760c0 con 0x7f47ec06ec20 2024-04-26T21:09:40.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.373+0000 7f47e8c00640 1 -- 172.21.15.26:0/1090570096 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47dc01a070 con 0x7f47ec06ec20 2024-04-26T21:09:40.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.374+0000 7f47e8c00640 1 --2- 172.21.15.26:0/1090570096 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f47c806c8f0 0x7f47c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:40.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.374+0000 7f47e8c00640 1 -- 172.21.15.26:0/1090570096 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4943+0+0 (secure 0 0 0) 0x7f47dc08b7f0 con 0x7f47ec06ec20 2024-04-26T21:09:40.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.374+0000 7f47e3e00640 1 --2- 172.21.15.26:0/1090570096 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f47c806c8f0 0x7f47c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:40.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.375+0000 7f47e3e00640 1 --2- 172.21.15.26:0/1090570096 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f47c806c8f0 0x7f47c806edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f47d4034000 tx=0x7f47d402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:40.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.379+0000 7f47e8c00640 1 -- 172.21.15.26:0/1090570096 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f47ec0760c0 con 0x7f47ec06ec20 2024-04-26T21:09:40.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:40.600+0000 7f47f2c00640 1 -- 172.21.15.26:0/1090570096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f47ec13b5c0 con 0x7f47ec06ec20 2024-04-26T21:09:41.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.098+0000 7f47e8c00640 1 -- 172.21.15.26:0/1090570096 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) ==== 97+0+0 (secure 0 0 0) 0x7f47dc059e80 con 0x7f47ec06ec20 2024-04-26T21:09:41.099 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-04-26T21:09:41.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.102+0000 7f47f2c00640 1 -- 172.21.15.26:0/1090570096 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f47c806c8f0 msgr2=0x7f47c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:41.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.102+0000 7f47f2c00640 1 --2- 172.21.15.26:0/1090570096 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f47c806c8f0 0x7f47c806edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f47d4034000 tx=0x7f47d402e000 comp rx=0 tx=0).stop 2024-04-26T21:09:41.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.102+0000 7f47f2c00640 1 -- 172.21.15.26:0/1090570096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ec06ec20 msgr2=0x7f47ec160600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:41.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.102+0000 7f47f2c00640 1 --2- 172.21.15.26:0/1090570096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ec06ec20 0x7f47ec160600 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00e8b0 tx=0x7f47dc00ed70 comp rx=0 tx=0).stop 2024-04-26T21:09:41.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.102+0000 7f47f2c00640 1 -- 172.21.15.26:0/1090570096 shutdown_connections 2024-04-26T21:09:41.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.102+0000 7f47f2c00640 1 --2- 172.21.15.26:0/1090570096 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f47c806c8f0 0x7f47c806edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.102+0000 7f47f2c00640 1 --2- 172.21.15.26:0/1090570096 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec146990 0x7f47ec1f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.102+0000 7f47f2c00640 1 --2- 172.21.15.26:0/1090570096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec0d1210 0x7f47ec1f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.102+0000 7f47f2c00640 1 --2- 172.21.15.26:0/1090570096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ec06ec20 0x7f47ec160600 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.102+0000 7f47f2c00640 1 -- 172.21.15.26:0/1090570096 >> 172.21.15.26:0/1090570096 conn(0x7f47ec0cd840 msgr2=0x7f47ec0b0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:41.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.103+0000 7f47f2c00640 1 -- 172.21.15.26:0/1090570096 shutdown_connections 2024-04-26T21:09:41.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.103+0000 7f47f2c00640 1 -- 172.21.15.26:0/1090570096 wait complete. 2024-04-26T21:09:41.127 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T21:09:41.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.323+0000 7f1566a00640 1 Processor -- start 2024-04-26T21:09:41.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.323+0000 7f1566a00640 1 -- start start 2024-04-26T21:09:41.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.325+0000 7f1566a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1560146830 0x7f156015d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:41.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.325+0000 7f1566a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f15600b2e30 0x7f15600b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:41.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.325+0000 7f1566a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15600b4280 con 0x7f15600b3d10 2024-04-26T21:09:41.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.325+0000 7f1566a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1560146c10 con 0x7f15600b2e30 2024-04-26T21:09:41.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.325+0000 7f1566a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f15600b4560 con 0x7f1560146830 2024-04-26T21:09:41.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.325+0000 7f1564c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f15600b3d10 0x7f1560073f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32876/0 (socket says 172.21.15.26:32876) 2024-04-26T21:09:41.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.325+0000 7f1564c00640 1 -- 172.21.15.26:0/1333800008 learned_addr learned my addr 172.21.15.26:0/1333800008 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:41.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.326+0000 7f1566000640 1 --2- 172.21.15.26:0/1333800008 >> v2:172.21.15.86:3300/0 conn(0x7f15600b2e30 0x7f15600b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:41.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.326+0000 7f155fe00640 1 -- 172.21.15.26:0/1333800008 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 640131723 0 0) 0x7f15600b4280 con 0x7f15600b3d10 2024-04-26T21:09:41.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.326+0000 7f1565600640 1 --2- 172.21.15.26:0/1333800008 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1560146830 0x7f156015d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:41.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.326+0000 7f155fe00640 1 -- 172.21.15.26:0/1333800008 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1548003520 con 0x7f15600b3d10 2024-04-26T21:09:41.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.326+0000 7f155fe00640 1 -- 172.21.15.26:0/1333800008 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 933059099 0 0) 0x7f1548003520 con 0x7f15600b3d10 2024-04-26T21:09:41.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.326+0000 7f155fe00640 1 -- 172.21.15.26:0/1333800008 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1560146830 msgr2=0x7f156015d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:41.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.326+0000 7f155fe00640 1 --2- 172.21.15.26:0/1333800008 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1560146830 0x7f156015d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.326+0000 7f155fe00640 1 -- 172.21.15.26:0/1333800008 >> v2:172.21.15.86:3300/0 conn(0x7f15600b2e30 msgr2=0x7f15600b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:41.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.326+0000 7f155fe00640 1 --2- 172.21.15.26:0/1333800008 >> v2:172.21.15.86:3300/0 conn(0x7f15600b2e30 0x7f15600b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.326+0000 7f155fe00640 1 -- 172.21.15.26:0/1333800008 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15600b4840 con 0x7f15600b3d10 2024-04-26T21:09:41.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.326+0000 7f155fe00640 1 -- 172.21.15.26:0/1333800008 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f154c001670 con 0x7f15600b3d10 2024-04-26T21:09:41.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.326+0000 7f155fe00640 1 -- 172.21.15.26:0/1333800008 >> v1:172.21.15.26:6789/0 conn(0x7f15600b3d10 legacy=0x7f1560073f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:41.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.326+0000 7f155fe00640 1 --2- 172.21.15.26:0/1333800008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15480039b0 0x7f1548003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:41.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.326+0000 7f155fe00640 1 --2- 172.21.15.26:0/1333800008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1548004800 0x7f1548004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:41.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.326+0000 7f155fe00640 1 --2- 172.21.15.26:0/1333800008 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1560146830 0x7f15480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:41.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.326+0000 7f155fe00640 1 -- 172.21.15.26:0/1333800008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1560146c10 con 0x7f15480039b0 2024-04-26T21:09:41.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.326+0000 7f155fe00640 1 -- 172.21.15.26:0/1333800008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f15600b4560 con 0x7f1548004800 2024-04-26T21:09:41.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.326+0000 7f155fe00640 1 -- 172.21.15.26:0/1333800008 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f154800ba10 con 0x7f1560146830 2024-04-26T21:09:41.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.327+0000 7f1565600640 1 --2- 172.21.15.26:0/1333800008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15480039b0 0x7f1548003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:41.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.327+0000 7f1564c00640 1 --2- 172.21.15.26:0/1333800008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1548004800 0x7f1548004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:41.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.327+0000 7f1566000640 1 --2- 172.21.15.26:0/1333800008 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1560146830 0x7f15480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:41.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.327+0000 7f1565600640 1 -- 172.21.15.26:0/1333800008 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1560146830 msgr2=0x7f15480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:41.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.327+0000 7f1565600640 1 --2- 172.21.15.26:0/1333800008 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1560146830 0x7f15480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.327+0000 7f1565600640 1 -- 172.21.15.26:0/1333800008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1548004800 msgr2=0x7f1548004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:41.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.327+0000 7f1565600640 1 --2- 172.21.15.26:0/1333800008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1548004800 0x7f1548004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.327+0000 7f1565600640 1 -- 172.21.15.26:0/1333800008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1548003520 con 0x7f15480039b0 2024-04-26T21:09:41.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.327+0000 7f1566000640 1 --2- 172.21.15.26:0/1333800008 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1560146830 0x7f15480051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:41.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.329+0000 7f1565600640 1 --2- 172.21.15.26:0/1333800008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15480039b0 0x7f1548003dd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f155400ba30 tx=0x7f1548005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f75c6b93cbe7b2c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:41.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.329+0000 7f155fe00640 1 -- 172.21.15.26:0/1333800008 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554030070 con 0x7f15480039b0 2024-04-26T21:09:41.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.329+0000 7f155fe00640 1 -- 172.21.15.26:0/1333800008 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f155402ade0 con 0x7f15480039b0 2024-04-26T21:09:41.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.329+0000 7f155fe00640 1 -- 172.21.15.26:0/1333800008 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155400b600 con 0x7f15480039b0 2024-04-26T21:09:41.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.330+0000 7f1566a00640 1 -- 172.21.15.26:0/1333800008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15480039b0 msgr2=0x7f1548003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:41.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.330+0000 7f1566a00640 1 --2- 172.21.15.26:0/1333800008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15480039b0 0x7f1548003dd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f155400ba30 tx=0x7f1548005830 comp rx=0 tx=0).stop 2024-04-26T21:09:41.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.330+0000 7f1566000640 1 -- 172.21.15.26:0/1333800008 reap_dead start 2024-04-26T21:09:41.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.330+0000 7f1566a00640 1 -- 172.21.15.26:0/1333800008 shutdown_connections 2024-04-26T21:09:41.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.330+0000 7f1566a00640 1 -- 172.21.15.26:0/1333800008 >> 172.21.15.26:0/1333800008 conn(0x7f156006c660 msgr2=0x7f156006ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:41.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.330+0000 7f1566a00640 1 -- 172.21.15.26:0/1333800008 shutdown_connections 2024-04-26T21:09:41.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.330+0000 7f1566a00640 1 -- 172.21.15.26:0/1333800008 wait complete. 2024-04-26T21:09:41.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.332+0000 7f1566a00640 1 Processor -- start 2024-04-26T21:09:41.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.332+0000 7f1566a00640 1 -- start start 2024-04-26T21:09:41.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.333+0000 7f1566a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15480039b0 0x7f15601f8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:41.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.333+0000 7f1566a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15600cb210 0x7f156015c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:41.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.333+0000 7f1566a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1560141d70 0x7f1560144160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:41.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.333+0000 7f1566a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f156015d6b0 con 0x7f15480039b0 2024-04-26T21:09:41.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.333+0000 7f1566a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f15600b4e00 con 0x7f15600cb210 2024-04-26T21:09:41.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.333+0000 7f1566a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f156015d830 con 0x7f1560141d70 2024-04-26T21:09:41.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.333+0000 7f1565600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15480039b0 0x7f15601f8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:41.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.333+0000 7f1565600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15480039b0 0x7f15601f8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47934/0 (socket says 172.21.15.26:47934) 2024-04-26T21:09:41.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.333+0000 7f1565600640 1 -- 172.21.15.26:0/4269476662 learned_addr learned my addr 172.21.15.26:0/4269476662 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:41.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.333+0000 7f1564c00640 1 --2- 172.21.15.26:0/4269476662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15600cb210 0x7f156015c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:41.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.333+0000 7f1566000640 1 --2- 172.21.15.26:0/4269476662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1560141d70 0x7f1560144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:41.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.333+0000 7f1565600640 1 -- 172.21.15.26:0/4269476662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1560141d70 msgr2=0x7f1560144160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:41.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.334+0000 7f1565600640 1 --2- 172.21.15.26:0/4269476662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1560141d70 0x7f1560144160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.334+0000 7f1565600640 1 -- 172.21.15.26:0/4269476662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15600cb210 msgr2=0x7f156015c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:41.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.334+0000 7f1565600640 1 --2- 172.21.15.26:0/4269476662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15600cb210 0x7f156015c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.334+0000 7f1565600640 1 -- 172.21.15.26:0/4269476662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15601f8d50 con 0x7f15480039b0 2024-04-26T21:09:41.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.334+0000 7f1564c00640 1 --2- 172.21.15.26:0/4269476662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15600cb210 0x7f156015c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:41.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.334+0000 7f1566000640 1 --2- 172.21.15.26:0/4269476662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1560141d70 0x7f1560144160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:41.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.334+0000 7f1565600640 1 --2- 172.21.15.26:0/4269476662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15480039b0 0x7f15601f8750 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f155400bb20 tx=0x7f15540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:41.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.334+0000 7f155d600640 1 -- 172.21.15.26:0/4269476662 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554043030 con 0x7f15480039b0 2024-04-26T21:09:41.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.334+0000 7f155d600640 1 -- 172.21.15.26:0/4269476662 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f155402abe0 con 0x7f15480039b0 2024-04-26T21:09:41.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.334+0000 7f155d600640 1 -- 172.21.15.26:0/4269476662 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155400b600 con 0x7f15480039b0 2024-04-26T21:09:41.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.334+0000 7f1566a00640 1 -- 172.21.15.26:0/4269476662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15601f9030 con 0x7f15480039b0 2024-04-26T21:09:41.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.334+0000 7f1566a00640 1 -- 172.21.15.26:0/4269476662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15601f95f0 con 0x7f15480039b0 2024-04-26T21:09:41.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.336+0000 7f1566a00640 1 -- 172.21.15.26:0/4269476662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f156006a700 con 0x7f15480039b0 2024-04-26T21:09:41.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.337+0000 7f155d600640 1 -- 172.21.15.26:0/4269476662 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1554046070 con 0x7f15480039b0 2024-04-26T21:09:41.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.338+0000 7f155d600640 1 --2- 172.21.15.26:0/4269476662 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f154006c8f0 0x7f154006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:41.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.338+0000 7f155d600640 1 -- 172.21.15.26:0/4269476662 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f15540ac260 con 0x7f15480039b0 2024-04-26T21:09:41.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.338+0000 7f1564c00640 1 --2- 172.21.15.26:0/4269476662 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f154006c8f0 0x7f154006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:41.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.339+0000 7f1564c00640 1 --2- 172.21.15.26:0/4269476662 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f154006c8f0 0x7f154006edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f154c000c00 tx=0x7f154c000ff0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:41.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.342+0000 7f155d600640 1 -- 172.21.15.26:0/4269476662 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f155403c3a0 con 0x7f15480039b0 2024-04-26T21:09:41.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.627+0000 7f1566a00640 1 -- 172.21.15.26:0/4269476662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f15600b3740 con 0x7f15480039b0 2024-04-26T21:09:41.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.628+0000 7f155d600640 1 -- 172.21.15.26:0/4269476662 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f155407a950 con 0x7f15480039b0 2024-04-26T21:09:41.629 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:41.629 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-26T21:09:41.629 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-04-26T21:09:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-04-26T21:09:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCcFyxmF64cEhAAsY4vNZPKu0uJ4m3oQEeKHQ==", 2024-04-26T21:09:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-26T21:09:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-26T21:09:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-26T21:09:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-26T21:09:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-04-26T21:09:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCcFyxmogPzLRAAcr843qKaNdncjkb48AIVrw==", 2024-04-26T21:09:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-26T21:09:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-26T21:09:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-26T21:09:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-26T21:09:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-04-26T21:09:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCcFyxmF2/mHxAA3c0MGKp8Cp7DKfxfztuzuA==", 2024-04-26T21:09:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-26T21:09:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-26T21:09:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-26T21:09:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-26T21:09:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-04-26T21:09:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCcFyxm6MwoOxAAnMV9jsem6T5o6Ixu4Mu5eA==", 2024-04-26T21:09:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-26T21:09:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-26T21:09:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-26T21:09:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-26T21:09:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-04-26T21:09:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCfFyxmtYeWExAA6tT0LS6D6NiVvIclusiezQ==", 2024-04-26T21:09:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-26T21:09:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-26T21:09:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-26T21:09:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-04-26T21:09:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCiFyxmYPqALhAAc3wZFz3UPv1auC3/sHsOnA==", 2024-04-26T21:09:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-26T21:09:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-26T21:09:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-26T21:09:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-04-26T21:09:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQClFyxmbNFUOxAA6tfTcUN+QfErkcilZwuz/w==", 2024-04-26T21:09:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-26T21:09:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-26T21:09:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-26T21:09:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-04-26T21:09:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCpFyxmYzPLGxAAJ6xcdW45JCAUyr2cy7FVxA==", 2024-04-26T21:09:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-26T21:09:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-26T21:09:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-26T21:09:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-04-26T21:09:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCvFyxmkjJEORAARc8N9amUPWZ9WhEpUJFVow==", 2024-04-26T21:09:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-26T21:09:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-26T21:09:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-26T21:09:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-04-26T21:09:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCzFyxmEuI4CBAAOGXURY52av9wIRssjinNXQ==", 2024-04-26T21:09:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-26T21:09:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-26T21:09:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-26T21:09:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-04-26T21:09:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC2Fyxm4lyNFBAAU/dYFNPNZIWF/P2KwM7WyA==", 2024-04-26T21:09:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-26T21:09:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-26T21:09:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-26T21:09:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-04-26T21:09:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5FyxmkNS2HhAAvCMZaOzs4ouX7YKsztShAw==", 2024-04-26T21:09:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-04-26T21:09:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-04-26T21:09:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-26T21:09:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-04-26T21:09:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCdFyxmV7X3ChAArBm2KV2Tw+YKfYxawc6g2A==", 2024-04-26T21:09:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-26T21:09:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-26T21:09:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-26T21:09:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-26T21:09:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-04-26T21:09:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCdFyxmHtm0HRAAow48USaeLV5a06ZQH9XRSA==", 2024-04-26T21:09:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-26T21:09:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-26T21:09:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-26T21:09:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-26T21:09:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.639 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.639 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.639 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2024-04-26T21:09:41.639 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCdFyxms6aUExAA5rF88r3JGr9aow2iyPzkoQ==", 2024-04-26T21:09:41.639 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.639 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-26T21:09:41.639 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-26T21:09:41.639 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-26T21:09:41.639 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-26T21:09:41.639 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.639 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.639 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.639 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2024-04-26T21:09:41.639 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCdFyxmlniDJhAAzqTQ10YbKBgioFqgCWBRmA==", 2024-04-26T21:09:41.640 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.640 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-04-26T21:09:41.640 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-04-26T21:09:41.640 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-04-26T21:09:41.640 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-04-26T21:09:41.640 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.640 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.640 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.640 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-04-26T21:09:41.640 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCbFyxme+zoERAAzX2oH3McFuKYnWA8YnI3Xw==", 2024-04-26T21:09:41.640 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.640 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-04-26T21:09:41.640 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-04-26T21:09:41.640 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-04-26T21:09:41.640 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-26T21:09:41.641 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.641 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.641 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.641 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-mds", 2024-04-26T21:09:41.641 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDCFyxmAQrcBhAA8wcNRNuslftkGHcOdhxlNw==", 2024-04-26T21:09:41.641 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.641 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-mds" 2024-04-26T21:09:41.641 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.641 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.641 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.641 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-mgr", 2024-04-26T21:09:41.641 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDCFyxmmSbcBhAAD9n6PPrIYrBrgDzVGKRbUQ==", 2024-04-26T21:09:41.641 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.641 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-26T21:09:41.641 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.641 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.642 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.642 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-osd", 2024-04-26T21:09:41.642 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDCFyxmgT/cBhAAprvN3yh7wKHdJ+Lrh6oKfw==", 2024-04-26T21:09:41.642 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.642 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-osd" 2024-04-26T21:09:41.642 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.642 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.642 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.642 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-rbd", 2024-04-26T21:09:41.642 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDCFyxmA1jcBhAALMV1DO+Fo/youGFpcAGpxA==", 2024-04-26T21:09:41.643 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.643 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-26T21:09:41.643 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.643 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.643 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.643 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-26T21:09:41.643 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDCFyxmDXHcBhAActc58Ky8gTo0V5C5Mw3JBg==", 2024-04-26T21:09:41.643 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.643 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-26T21:09:41.643 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.643 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.644 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.644 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-rgw", 2024-04-26T21:09:41.644 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDCFyxm147cBhAAZU4jTM8ieddqoYTM9cnghg==", 2024-04-26T21:09:41.644 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.644 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-26T21:09:41.644 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.644 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.644 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.644 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-04-26T21:09:41.644 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCcFyxmEmYLCBAAkTnC04uqZ14rY5bgpM9PDw==", 2024-04-26T21:09:41.644 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.644 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-04-26T21:09:41.645 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-04-26T21:09:41.645 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-26T21:09:41.645 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.645 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:09:41.645 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:41.645 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-04-26T21:09:41.645 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCbFyxmOq38OhAAj7hQe7iTiydqLskFEDsUeA==", 2024-04-26T21:09:41.645 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-04-26T21:09:41.645 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-04-26T21:09:41.645 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-04-26T21:09:41.645 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-04-26T21:09:41.645 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.645 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:41.645 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-26T21:09:41.645 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-26T21:09:41.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.631+0000 7f1566a00640 1 -- 172.21.15.26:0/4269476662 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f154006c8f0 msgr2=0x7f154006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:41.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.631+0000 7f1566a00640 1 --2- 172.21.15.26:0/4269476662 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f154006c8f0 0x7f154006edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f154c000c00 tx=0x7f154c000ff0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.631+0000 7f1566a00640 1 -- 172.21.15.26:0/4269476662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15480039b0 msgr2=0x7f15601f8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.631+0000 7f1566a00640 1 --2- 172.21.15.26:0/4269476662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15480039b0 0x7f15601f8750 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f155400bb20 tx=0x7f15540046c0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.631+0000 7f1566a00640 1 -- 172.21.15.26:0/4269476662 shutdown_connections 2024-04-26T21:09:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.631+0000 7f1566a00640 1 --2- 172.21.15.26:0/4269476662 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f154006c8f0 0x7f154006edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.631+0000 7f1566a00640 1 --2- 172.21.15.26:0/4269476662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1560141d70 0x7f1560144160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.631+0000 7f1566a00640 1 --2- 172.21.15.26:0/4269476662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15600cb210 0x7f156015c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.631+0000 7f1566a00640 1 --2- 172.21.15.26:0/4269476662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15480039b0 0x7f15601f8750 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.631+0000 7f1566a00640 1 -- 172.21.15.26:0/4269476662 >> 172.21.15.26:0/4269476662 conn(0x7f156006c660 msgr2=0x7f15600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.631+0000 7f1566a00640 1 -- 172.21.15.26:0/4269476662 shutdown_connections 2024-04-26T21:09:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.631+0000 7f1566a00640 1 -- 172.21.15.26:0/4269476662 wait complete. 2024-04-26T21:09:41.651 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-26T21:09:41.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.863+0000 7fc49b000640 1 Processor -- start 2024-04-26T21:09:41.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.863+0000 7fc49b000640 1 -- start start 2024-04-26T21:09:41.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.864+0000 7fc49b000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc494146990 0x7fc4940b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:41.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.864+0000 7fc49b000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc4940d13e0 0x7fc494072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:41.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.864+0000 7fc49b000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4940b4480 con 0x7fc49406ec20 2024-04-26T21:09:41.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.864+0000 7fc49b000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fc494146d70 con 0x7fc494146990 2024-04-26T21:09:41.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.864+0000 7fc49b000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc494073b70 con 0x7fc4940d13e0 2024-04-26T21:09:41.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.865+0000 7fc493e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc494146990 0x7fc4940b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:41.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.865+0000 7fc493e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc494146990 0x7fc4940b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47942/0 (socket says 172.21.15.26:47942) 2024-04-26T21:09:41.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.865+0000 7fc493e00640 1 -- 172.21.15.26:0/3670352843 learned_addr learned my addr 172.21.15.26:0/3670352843 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:41.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.865+0000 7fc499000640 1 --2- 172.21.15.26:0/3670352843 >> v2:172.21.15.86:3300/0 conn(0x7fc4940d13e0 0x7fc494072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:41.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.865+0000 7fc493e00640 1 -- 172.21.15.26:0/3670352843 >> v2:172.21.15.86:3300/0 conn(0x7fc4940d13e0 msgr2=0x7fc494072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:41.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.865+0000 7fc493e00640 1 --2- 172.21.15.26:0/3670352843 >> v2:172.21.15.86:3300/0 conn(0x7fc4940d13e0 0x7fc494072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.865+0000 7fc493e00640 1 -- 172.21.15.26:0/3670352843 >> v1:172.21.15.26:6789/0 conn(0x7fc49406ec20 legacy=0x7fc4940d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:41.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.865+0000 7fc493e00640 1 -- 172.21.15.26:0/3670352843 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc494073e50 con 0x7fc494146990 2024-04-26T21:09:41.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.865+0000 7fc499000640 1 --2- 172.21.15.26:0/3670352843 >> v2:172.21.15.86:3300/0 conn(0x7fc4940d13e0 0x7fc494072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:41.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.866+0000 7fc493e00640 1 --2- 172.21.15.26:0/3670352843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc494146990 0x7fc4940b3f40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc494070610 tx=0x7fc48402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2618ce95b51b0b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:41.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.867+0000 7fc492a00640 1 -- 172.21.15.26:0/3670352843 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484037070 con 0x7fc494146990 2024-04-26T21:09:41.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.867+0000 7fc492a00640 1 -- 172.21.15.26:0/3670352843 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc484032440 con 0x7fc494146990 2024-04-26T21:09:41.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.867+0000 7fc492a00640 1 -- 172.21.15.26:0/3670352843 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484032810 con 0x7fc494146990 2024-04-26T21:09:41.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.867+0000 7fc49b000640 1 -- 172.21.15.26:0/3670352843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc494146990 msgr2=0x7fc4940b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:41.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.867+0000 7fc49b000640 1 --2- 172.21.15.26:0/3670352843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc494146990 0x7fc4940b3f40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc494070610 tx=0x7fc48402a270 comp rx=0 tx=0).stop 2024-04-26T21:09:41.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.867+0000 7fc49b000640 1 -- 172.21.15.26:0/3670352843 shutdown_connections 2024-04-26T21:09:41.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.867+0000 7fc49b000640 1 --2- 172.21.15.26:0/3670352843 >> v2:172.21.15.86:3300/0 conn(0x7fc4940d13e0 0x7fc494072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.867+0000 7fc49b000640 1 --2- 172.21.15.26:0/3670352843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc494146990 0x7fc4940b3f40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.867+0000 7fc49b000640 1 -- 172.21.15.26:0/3670352843 >> 172.21.15.26:0/3670352843 conn(0x7fc4940cd840 msgr2=0x7fc4940cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:41.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.867+0000 7fc49b000640 1 -- 172.21.15.26:0/3670352843 shutdown_connections 2024-04-26T21:09:41.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.867+0000 7fc49b000640 1 -- 172.21.15.26:0/3670352843 wait complete. 2024-04-26T21:09:41.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.869+0000 7fc49b000640 1 Processor -- start 2024-04-26T21:09:41.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.869+0000 7fc49b000640 1 -- start start 2024-04-26T21:09:41.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.870+0000 7fc49b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc49406ec20 0x7fc494159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:41.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.870+0000 7fc49b000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4940d13e0 0x7fc494158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:41.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.870+0000 7fc49b000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc494146990 0x7fc4941593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:41.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.870+0000 7fc49b000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc49406e700 con 0x7fc4940d13e0 2024-04-26T21:09:41.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.870+0000 7fc49b000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc49406e420 con 0x7fc49406ec20 2024-04-26T21:09:41.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.870+0000 7fc49b000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc49406f0b0 con 0x7fc494146990 2024-04-26T21:09:41.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.871+0000 7fc493400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4940d13e0 0x7fc494158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:41.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.871+0000 7fc493400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4940d13e0 0x7fc494158670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47948/0 (socket says 172.21.15.26:47948) 2024-04-26T21:09:41.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.871+0000 7fc493400640 1 -- 172.21.15.26:0/890746271 learned_addr learned my addr 172.21.15.26:0/890746271 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:41.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.871+0000 7fc499000640 1 --2- 172.21.15.26:0/890746271 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc494146990 0x7fc4941593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:41.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.871+0000 7fc493e00640 1 --2- 172.21.15.26:0/890746271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc49406ec20 0x7fc494159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:41.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.871+0000 7fc493400640 1 -- 172.21.15.26:0/890746271 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc494146990 msgr2=0x7fc4941593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:41.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.871+0000 7fc493400640 1 --2- 172.21.15.26:0/890746271 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc494146990 0x7fc4941593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.871+0000 7fc493400640 1 -- 172.21.15.26:0/890746271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc49406ec20 msgr2=0x7fc494159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:41.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.871+0000 7fc493400640 1 --2- 172.21.15.26:0/890746271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc49406ec20 0x7fc494159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:41.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.871+0000 7fc493400640 1 -- 172.21.15.26:0/890746271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49415a150 con 0x7fc4940d13e0 2024-04-26T21:09:41.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.871+0000 7fc493e00640 1 --2- 172.21.15.26:0/890746271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc49406ec20 0x7fc494159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:41.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.872+0000 7fc493400640 1 --2- 172.21.15.26:0/890746271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4940d13e0 0x7fc494158670 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc48000c3f0 tx=0x7fc48000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:41.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.872+0000 7fc48be00640 1 -- 172.21.15.26:0/890746271 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4800052d0 con 0x7fc4940d13e0 2024-04-26T21:09:41.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.872+0000 7fc48be00640 1 -- 172.21.15.26:0/890746271 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc480005490 con 0x7fc4940d13e0 2024-04-26T21:09:41.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.872+0000 7fc49b000640 1 -- 172.21.15.26:0/890746271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49415a430 con 0x7fc4940d13e0 2024-04-26T21:09:41.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.872+0000 7fc48be00640 1 -- 172.21.15.26:0/890746271 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480007260 con 0x7fc4940d13e0 2024-04-26T21:09:41.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.872+0000 7fc49b000640 1 -- 172.21.15.26:0/890746271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc494157e70 con 0x7fc4940d13e0 2024-04-26T21:09:41.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.873+0000 7fc49b000640 1 -- 172.21.15.26:0/890746271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc49406a700 con 0x7fc4940d13e0 2024-04-26T21:09:41.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.874+0000 7fc48be00640 1 -- 172.21.15.26:0/890746271 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc480007400 con 0x7fc4940d13e0 2024-04-26T21:09:41.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.875+0000 7fc48be00640 1 --2- 172.21.15.26:0/890746271 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc47406c8f0 0x7fc47406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:41.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.875+0000 7fc48be00640 1 -- 172.21.15.26:0/890746271 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fc48008ad60 con 0x7fc4940d13e0 2024-04-26T21:09:41.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.876+0000 7fc493e00640 1 --2- 172.21.15.26:0/890746271 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc47406c8f0 0x7fc47406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:41.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.876+0000 7fc493e00640 1 --2- 172.21.15.26:0/890746271 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc47406c8f0 0x7fc47406edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc484034000 tx=0x7fc48402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:41.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:41.880+0000 7fc48be00640 1 -- 172.21.15.26:0/890746271 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc480059440 con 0x7fc4940d13e0 2024-04-26T21:09:42.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.166+0000 7fc49b000640 1 -- 172.21.15.26:0/890746271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fc494145810 con 0x7fc4940d13e0 2024-04-26T21:09:42.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.171+0000 7fc48be00640 1 -- 172.21.15.26:0/890746271 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fc480058d40 con 0x7fc4940d13e0 2024-04-26T21:09:42.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.174+0000 7fc49b000640 1 -- 172.21.15.26:0/890746271 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc47406c8f0 msgr2=0x7fc47406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.174+0000 7fc49b000640 1 --2- 172.21.15.26:0/890746271 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc47406c8f0 0x7fc47406edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc484034000 tx=0x7fc48402e000 comp rx=0 tx=0).stop 2024-04-26T21:09:42.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.174+0000 7fc49b000640 1 -- 172.21.15.26:0/890746271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4940d13e0 msgr2=0x7fc494158670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.174+0000 7fc49b000640 1 --2- 172.21.15.26:0/890746271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4940d13e0 0x7fc494158670 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc48000c3f0 tx=0x7fc48000c8b0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.174+0000 7fc49b000640 1 -- 172.21.15.26:0/890746271 shutdown_connections 2024-04-26T21:09:42.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.174+0000 7fc49b000640 1 --2- 172.21.15.26:0/890746271 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc47406c8f0 0x7fc47406edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.174+0000 7fc49b000640 1 --2- 172.21.15.26:0/890746271 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc494146990 0x7fc4941593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.174+0000 7fc49b000640 1 --2- 172.21.15.26:0/890746271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4940d13e0 0x7fc494158670 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.174+0000 7fc49b000640 1 --2- 172.21.15.26:0/890746271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc49406ec20 0x7fc494159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.174+0000 7fc49b000640 1 -- 172.21.15.26:0/890746271 >> 172.21.15.26:0/890746271 conn(0x7fc4940cd840 msgr2=0x7fc494072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:42.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.174+0000 7fc49b000640 1 -- 172.21.15.26:0/890746271 shutdown_connections 2024-04-26T21:09:42.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.175+0000 7fc49b000640 1 -- 172.21.15.26:0/890746271 wait complete. 2024-04-26T21:09:42.194 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-26T21:09:42.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.401+0000 7fc281800640 1 Processor -- start 2024-04-26T21:09:42.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.401+0000 7fc281800640 1 -- start start 2024-04-26T21:09:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.403+0000 7fc281800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc27c0b3d10 0x7fc27c15d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.403+0000 7fc281800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc27c0b2e30 0x7fc27c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.403+0000 7fc281800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc27c0b4280 con 0x7fc27c146830 2024-04-26T21:09:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.403+0000 7fc281800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc27c146c10 con 0x7fc27c0b2e30 2024-04-26T21:09:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.403+0000 7fc281800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc27c0b4560 con 0x7fc27c0b3d10 2024-04-26T21:09:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.403+0000 7fc27be00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc27c146830 0x7fc27c073f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32896/0 (socket says 172.21.15.26:32896) 2024-04-26T21:09:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.403+0000 7fc27be00640 1 -- 172.21.15.26:0/1436944838 learned_addr learned my addr 172.21.15.26:0/1436944838 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.404+0000 7fc27aa00640 1 -- 172.21.15.26:0/1436944838 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 124178552 0 0) 0x7fc27c0b4280 con 0x7fc27c146830 2024-04-26T21:09:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.404+0000 7fc27aa00640 1 -- 172.21.15.26:0/1436944838 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc260003520 con 0x7fc27c146830 2024-04-26T21:09:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.404+0000 7fc27b400640 1 --2- 172.21.15.26:0/1436944838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc27c0b3d10 0x7fc27c15d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.404+0000 7fc280e00640 1 --2- 172.21.15.26:0/1436944838 >> v2:172.21.15.86:3300/0 conn(0x7fc27c0b2e30 0x7fc27c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.404+0000 7fc27aa00640 1 -- 172.21.15.26:0/1436944838 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1098135170 0 0) 0x7fc260003520 con 0x7fc27c146830 2024-04-26T21:09:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.404+0000 7fc27aa00640 1 -- 172.21.15.26:0/1436944838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc27c0b3d10 msgr2=0x7fc27c15d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.404+0000 7fc27aa00640 1 --2- 172.21.15.26:0/1436944838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc27c0b3d10 0x7fc27c15d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.404+0000 7fc27aa00640 1 -- 172.21.15.26:0/1436944838 >> v2:172.21.15.86:3300/0 conn(0x7fc27c0b2e30 msgr2=0x7fc27c0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.404+0000 7fc27aa00640 1 --2- 172.21.15.26:0/1436944838 >> v2:172.21.15.86:3300/0 conn(0x7fc27c0b2e30 0x7fc27c0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.404+0000 7fc27aa00640 1 -- 172.21.15.26:0/1436944838 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27c0b4840 con 0x7fc27c146830 2024-04-26T21:09:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.404+0000 7fc27aa00640 1 -- 172.21.15.26:0/1436944838 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7fc26c0019f0 con 0x7fc27c146830 2024-04-26T21:09:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.404+0000 7fc27aa00640 1 -- 172.21.15.26:0/1436944838 >> v1:172.21.15.26:6789/0 conn(0x7fc27c146830 legacy=0x7fc27c073f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.404+0000 7fc27aa00640 1 --2- 172.21.15.26:0/1436944838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2600039b0 0x7fc260003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.404+0000 7fc27aa00640 1 --2- 172.21.15.26:0/1436944838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc260004800 0x7fc260004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.404+0000 7fc27aa00640 1 --2- 172.21.15.26:0/1436944838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc27c0b3d10 0x7fc2600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.404+0000 7fc27aa00640 1 -- 172.21.15.26:0/1436944838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c146c10 con 0x7fc260004800 2024-04-26T21:09:42.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.404+0000 7fc27aa00640 1 -- 172.21.15.26:0/1436944838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c0b4560 con 0x7fc2600039b0 2024-04-26T21:09:42.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.404+0000 7fc27aa00640 1 -- 172.21.15.26:0/1436944838 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc26000baf0 con 0x7fc27c0b3d10 2024-04-26T21:09:42.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.405+0000 7fc27b400640 1 --2- 172.21.15.26:0/1436944838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc260004800 0x7fc260004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.405+0000 7fc27be00640 1 --2- 172.21.15.26:0/1436944838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2600039b0 0x7fc260003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.405+0000 7fc280e00640 1 --2- 172.21.15.26:0/1436944838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc27c0b3d10 0x7fc2600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.405+0000 7fc27b400640 1 -- 172.21.15.26:0/1436944838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc27c0b3d10 msgr2=0x7fc2600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:42.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.405+0000 7fc27b400640 1 --2- 172.21.15.26:0/1436944838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc27c0b3d10 0x7fc2600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.405+0000 7fc27b400640 1 -- 172.21.15.26:0/1436944838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2600039b0 msgr2=0x7fc260003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:42.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.405+0000 7fc27b400640 1 --2- 172.21.15.26:0/1436944838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2600039b0 0x7fc260003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.405+0000 7fc27b400640 1 -- 172.21.15.26:0/1436944838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc260003520 con 0x7fc260004800 2024-04-26T21:09:42.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.405+0000 7fc280e00640 1 --2- 172.21.15.26:0/1436944838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc27c0b3d10 0x7fc2600051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.406+0000 7fc27b400640 1 --2- 172.21.15.26:0/1436944838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc260004800 0x7fc260004c80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc26400b9e0 tx=0x7fc260005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6edca6c0f5596a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.406+0000 7fc27aa00640 1 -- 172.21.15.26:0/1436944838 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264030070 con 0x7fc260004800 2024-04-26T21:09:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.406+0000 7fc27aa00640 1 -- 172.21.15.26:0/1436944838 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc26402ad90 con 0x7fc260004800 2024-04-26T21:09:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.406+0000 7fc27aa00640 1 -- 172.21.15.26:0/1436944838 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26400b5e0 con 0x7fc260004800 2024-04-26T21:09:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.407+0000 7fc281800640 1 -- 172.21.15.26:0/1436944838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc260004800 msgr2=0x7fc260004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.407+0000 7fc281800640 1 --2- 172.21.15.26:0/1436944838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc260004800 0x7fc260004c80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc26400b9e0 tx=0x7fc260005830 comp rx=0 tx=0).stop 2024-04-26T21:09:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.407+0000 7fc280e00640 1 -- 172.21.15.26:0/1436944838 reap_dead start 2024-04-26T21:09:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.407+0000 7fc281800640 1 -- 172.21.15.26:0/1436944838 shutdown_connections 2024-04-26T21:09:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.407+0000 7fc281800640 1 -- 172.21.15.26:0/1436944838 >> 172.21.15.26:0/1436944838 conn(0x7fc27c06c660 msgr2=0x7fc27c06ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.407+0000 7fc281800640 1 -- 172.21.15.26:0/1436944838 shutdown_connections 2024-04-26T21:09:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.407+0000 7fc281800640 1 -- 172.21.15.26:0/1436944838 wait complete. 2024-04-26T21:09:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.409+0000 7fc281800640 1 Processor -- start 2024-04-26T21:09:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.409+0000 7fc281800640 1 -- start start 2024-04-26T21:09:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.410+0000 7fc281800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc260004800 0x7fc27c1400f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.410+0000 7fc281800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc27c13cf60 0x7fc27c13b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.410+0000 7fc281800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc27c06f830 0x7fc27c06fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.410+0000 7fc281800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c15d6b0 con 0x7fc260004800 2024-04-26T21:09:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.410+0000 7fc281800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c0b4e00 con 0x7fc27c06f830 2024-04-26T21:09:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.410+0000 7fc281800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc27c15d830 con 0x7fc27c13cf60 2024-04-26T21:09:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.410+0000 7fc27be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc260004800 0x7fc27c1400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.410+0000 7fc27be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc260004800 0x7fc27c1400f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47960/0 (socket says 172.21.15.26:47960) 2024-04-26T21:09:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.410+0000 7fc27be00640 1 -- 172.21.15.26:0/4207779448 learned_addr learned my addr 172.21.15.26:0/4207779448 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:42.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.411+0000 7fc280e00640 1 --2- 172.21.15.26:0/4207779448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc27c06f830 0x7fc27c06fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.411+0000 7fc27be00640 1 -- 172.21.15.26:0/4207779448 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc27c13cf60 msgr2=0x7fc27c13b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.411+0000 7fc27b400640 1 --2- 172.21.15.26:0/4207779448 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc27c13cf60 0x7fc27c13b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.411+0000 7fc27be00640 1 --2- 172.21.15.26:0/4207779448 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc27c13cf60 0x7fc27c13b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.411+0000 7fc27be00640 1 -- 172.21.15.26:0/4207779448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc27c06f830 msgr2=0x7fc27c06fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.411+0000 7fc27be00640 1 --2- 172.21.15.26:0/4207779448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc27c06f830 0x7fc27c06fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.411+0000 7fc27be00640 1 -- 172.21.15.26:0/4207779448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27c13b8b0 con 0x7fc260004800 2024-04-26T21:09:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.411+0000 7fc280e00640 1 --2- 172.21.15.26:0/4207779448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc27c06f830 0x7fc27c06fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.411+0000 7fc27be00640 1 --2- 172.21.15.26:0/4207779448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc260004800 0x7fc27c1400f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc26c007a30 tx=0x7fc26c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.411+0000 7fc273e00640 1 -- 172.21.15.26:0/4207779448 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c00edf0 con 0x7fc260004800 2024-04-26T21:09:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.411+0000 7fc273e00640 1 -- 172.21.15.26:0/4207779448 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc26c018070 con 0x7fc260004800 2024-04-26T21:09:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.411+0000 7fc273e00640 1 -- 172.21.15.26:0/4207779448 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c013640 con 0x7fc260004800 2024-04-26T21:09:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.411+0000 7fc281800640 1 -- 172.21.15.26:0/4207779448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27c13bb50 con 0x7fc260004800 2024-04-26T21:09:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.412+0000 7fc281800640 1 -- 172.21.15.26:0/4207779448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc27c13c110 con 0x7fc260004800 2024-04-26T21:09:42.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.413+0000 7fc281800640 1 -- 172.21.15.26:0/4207779448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc27c0701f0 con 0x7fc260004800 2024-04-26T21:09:42.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.414+0000 7fc273e00640 1 -- 172.21.15.26:0/4207779448 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc26c0137e0 con 0x7fc260004800 2024-04-26T21:09:42.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.415+0000 7fc273e00640 1 --2- 172.21.15.26:0/4207779448 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc25806c920 0x7fc25806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.415+0000 7fc273e00640 1 -- 172.21.15.26:0/4207779448 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fc26c08bf10 con 0x7fc260004800 2024-04-26T21:09:42.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.415+0000 7fc27b400640 1 --2- 172.21.15.26:0/4207779448 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc25806c920 0x7fc25806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.416+0000 7fc27b400640 1 --2- 172.21.15.26:0/4207779448 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc25806c920 0x7fc25806ede0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc260004780 tx=0x7fc264030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:42.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.420+0000 7fc273e00640 1 -- 172.21.15.26:0/4207779448 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc26c059fa0 con 0x7fc260004800 2024-04-26T21:09:42.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.691+0000 7fc281800640 1 -- 172.21.15.26:0/4207779448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fc27c06ce80 con 0x7fc260004800 2024-04-26T21:09:42.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.695+0000 7fc273e00640 1 -- 172.21.15.26:0/4207779448 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7fc26c05e2a0 con 0x7fc260004800 2024-04-26T21:09:42.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.697+0000 7fc281800640 1 -- 172.21.15.26:0/4207779448 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc25806c920 msgr2=0x7fc25806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.697+0000 7fc281800640 1 --2- 172.21.15.26:0/4207779448 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc25806c920 0x7fc25806ede0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc260004780 tx=0x7fc264030000 comp rx=0 tx=0).stop 2024-04-26T21:09:42.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.697+0000 7fc281800640 1 -- 172.21.15.26:0/4207779448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc260004800 msgr2=0x7fc27c1400f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.697+0000 7fc281800640 1 --2- 172.21.15.26:0/4207779448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc260004800 0x7fc27c1400f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc26c007a30 tx=0x7fc26c00e530 comp rx=0 tx=0).stop 2024-04-26T21:09:42.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.697+0000 7fc281800640 1 -- 172.21.15.26:0/4207779448 shutdown_connections 2024-04-26T21:09:42.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.698+0000 7fc281800640 1 --2- 172.21.15.26:0/4207779448 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc25806c920 0x7fc25806ede0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.698+0000 7fc281800640 1 --2- 172.21.15.26:0/4207779448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc27c06f830 0x7fc27c06fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.698+0000 7fc281800640 1 --2- 172.21.15.26:0/4207779448 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc27c13cf60 0x7fc27c13b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.698+0000 7fc281800640 1 --2- 172.21.15.26:0/4207779448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc260004800 0x7fc27c1400f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.698+0000 7fc281800640 1 -- 172.21.15.26:0/4207779448 >> 172.21.15.26:0/4207779448 conn(0x7fc27c06c660 msgr2=0x7fc27c15dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:42.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.698+0000 7fc281800640 1 -- 172.21.15.26:0/4207779448 shutdown_connections 2024-04-26T21:09:42.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.698+0000 7fc281800640 1 -- 172.21.15.26:0/4207779448 wait complete. 2024-04-26T21:09:42.716 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-26T21:09:42.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.917+0000 7f1513e00640 1 Processor -- start 2024-04-26T21:09:42.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.917+0000 7f1513e00640 1 -- start start 2024-04-26T21:09:42.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.918+0000 7f1513e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1514162740 0x7f151406fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.918+0000 7f1513e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1514160930 0x7f151406ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.918+0000 7f1513e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1514162b20 con 0x7f15140b2e30 2024-04-26T21:09:42.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.919+0000 7f1513e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f15140cd750 con 0x7f1514160930 2024-04-26T21:09:42.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.919+0000 7f1513e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f15140cda30 con 0x7f1514162740 2024-04-26T21:09:42.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.919+0000 7f1513400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f15140b2e30 0x7f15140ceb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32912/0 (socket says 172.21.15.26:32912) 2024-04-26T21:09:42.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.919+0000 7f1513400640 1 -- 172.21.15.26:0/460609908 learned_addr learned my addr 172.21.15.26:0/460609908 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:42.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.919+0000 7f1512a00640 1 --2- 172.21.15.26:0/460609908 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1514162740 0x7f151406fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.919+0000 7f1512000640 1 --2- 172.21.15.26:0/460609908 >> v2:172.21.15.86:3300/0 conn(0x7f1514160930 0x7f151406ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.920+0000 7f1511600640 1 -- 172.21.15.26:0/460609908 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2246758502 0 0) 0x7f1514162b20 con 0x7f15140b2e30 2024-04-26T21:09:42.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.920+0000 7f1511600640 1 -- 172.21.15.26:0/460609908 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14f8003520 con 0x7f15140b2e30 2024-04-26T21:09:42.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.920+0000 7f1511600640 1 -- 172.21.15.26:0/460609908 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 883537392 0 0) 0x7f14f8003520 con 0x7f15140b2e30 2024-04-26T21:09:42.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.920+0000 7f1511600640 1 -- 172.21.15.26:0/460609908 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1514162740 msgr2=0x7f151406fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:42.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.920+0000 7f1511600640 1 --2- 172.21.15.26:0/460609908 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1514162740 0x7f151406fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.920+0000 7f1511600640 1 -- 172.21.15.26:0/460609908 >> v2:172.21.15.86:3300/0 conn(0x7f1514160930 msgr2=0x7f151406ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:42.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.920+0000 7f1511600640 1 --2- 172.21.15.26:0/460609908 >> v2:172.21.15.86:3300/0 conn(0x7f1514160930 0x7f151406ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.920+0000 7f1511600640 1 -- 172.21.15.26:0/460609908 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15140cdd10 con 0x7f15140b2e30 2024-04-26T21:09:42.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.920+0000 7f1511600640 1 -- 172.21.15.26:0/460609908 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f150c0019f0 con 0x7f15140b2e30 2024-04-26T21:09:42.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.920+0000 7f1511600640 1 -- 172.21.15.26:0/460609908 >> v1:172.21.15.26:6789/0 conn(0x7f15140b2e30 legacy=0x7f15140ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.920+0000 7f1511600640 1 --2- 172.21.15.26:0/460609908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14f80039b0 0x7f14f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.920+0000 7f1511600640 1 --2- 172.21.15.26:0/460609908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14f8004800 0x7f14f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.920+0000 7f1511600640 1 --2- 172.21.15.26:0/460609908 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1514162740 0x7f14f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.920+0000 7f1511600640 1 -- 172.21.15.26:0/460609908 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15140cd750 con 0x7f14f8004800 2024-04-26T21:09:42.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.920+0000 7f1511600640 1 -- 172.21.15.26:0/460609908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f15140cda30 con 0x7f14f80039b0 2024-04-26T21:09:42.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.920+0000 7f1511600640 1 -- 172.21.15.26:0/460609908 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f14f800baa0 con 0x7f1514162740 2024-04-26T21:09:42.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.921+0000 7f1512000640 1 --2- 172.21.15.26:0/460609908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14f8004800 0x7f14f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.921+0000 7f1512a00640 1 --2- 172.21.15.26:0/460609908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14f80039b0 0x7f14f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.921+0000 7f1512000640 1 -- 172.21.15.26:0/460609908 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1514162740 msgr2=0x7f14f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:42.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.921+0000 7f1512000640 1 --2- 172.21.15.26:0/460609908 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1514162740 0x7f14f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.921+0000 7f1512000640 1 -- 172.21.15.26:0/460609908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14f80039b0 msgr2=0x7f14f8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:42.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.921+0000 7f1512000640 1 --2- 172.21.15.26:0/460609908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14f80039b0 0x7f14f8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.921+0000 7f1512000640 1 -- 172.21.15.26:0/460609908 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f8003520 con 0x7f14f8004800 2024-04-26T21:09:42.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.921+0000 7f1512a00640 1 --2- 172.21.15.26:0/460609908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14f80039b0 0x7f14f8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:42.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.922+0000 7f1512000640 1 --2- 172.21.15.26:0/460609908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14f8004800 0x7f14f8004c80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1514160d10 tx=0x7f14f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57aded01588fa778 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:42.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.922+0000 7f1511600640 1 -- 172.21.15.26:0/460609908 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500036070 con 0x7f14f8004800 2024-04-26T21:09:42.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.923+0000 7f1511600640 1 -- 172.21.15.26:0/460609908 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1500031400 con 0x7f14f8004800 2024-04-26T21:09:42.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.923+0000 7f1511600640 1 -- 172.21.15.26:0/460609908 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500009690 con 0x7f14f8004800 2024-04-26T21:09:42.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.923+0000 7f1513e00640 1 -- 172.21.15.26:0/460609908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14f8004800 msgr2=0x7f14f8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.923+0000 7f1513e00640 1 --2- 172.21.15.26:0/460609908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14f8004800 0x7f14f8004c80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1514160d10 tx=0x7f14f8005830 comp rx=0 tx=0).stop 2024-04-26T21:09:42.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.923+0000 7f1513400640 1 -- 172.21.15.26:0/460609908 reap_dead start 2024-04-26T21:09:42.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.923+0000 7f1513e00640 1 -- 172.21.15.26:0/460609908 shutdown_connections 2024-04-26T21:09:42.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.923+0000 7f1513e00640 1 -- 172.21.15.26:0/460609908 >> 172.21.15.26:0/460609908 conn(0x7f151406c660 msgr2=0x7f151406ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:42.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.923+0000 7f1513e00640 1 -- 172.21.15.26:0/460609908 shutdown_connections 2024-04-26T21:09:42.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.923+0000 7f1513e00640 1 -- 172.21.15.26:0/460609908 wait complete. 2024-04-26T21:09:42.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.925+0000 7f1513e00640 1 Processor -- start 2024-04-26T21:09:42.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.925+0000 7f1513e00640 1 -- start start 2024-04-26T21:09:42.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.926+0000 7f1513e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14f8004800 0x7f15140b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.926+0000 7f1513e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f151414c9b0 0x7f151415a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.926+0000 7f1513e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15141577c0 0x7f151415a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.926+0000 7f1513e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15140d0920 con 0x7f151414c9b0 2024-04-26T21:09:42.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.926+0000 7f1513e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f15140ce2d0 con 0x7f15141577c0 2024-04-26T21:09:42.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.926+0000 7f1513e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f15140d0aa0 con 0x7f14f8004800 2024-04-26T21:09:42.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.927+0000 7f1512000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f151414c9b0 0x7f151415a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.927+0000 7f1512000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f151414c9b0 0x7f151415a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47984/0 (socket says 172.21.15.26:47984) 2024-04-26T21:09:42.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.927+0000 7f1512000640 1 -- 172.21.15.26:0/2496531362 learned_addr learned my addr 172.21.15.26:0/2496531362 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:42.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.927+0000 7f1513400640 1 --2- 172.21.15.26:0/2496531362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15141577c0 0x7f151415a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.927+0000 7f1512a00640 1 --2- 172.21.15.26:0/2496531362 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14f8004800 0x7f15140b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.927+0000 7f1512000640 1 -- 172.21.15.26:0/2496531362 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14f8004800 msgr2=0x7f15140b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.927+0000 7f1512000640 1 --2- 172.21.15.26:0/2496531362 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14f8004800 0x7f15140b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.927+0000 7f1512000640 1 -- 172.21.15.26:0/2496531362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15141577c0 msgr2=0x7f151415a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:42.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.927+0000 7f1512000640 1 --2- 172.21.15.26:0/2496531362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15141577c0 0x7f151415a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:42.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.927+0000 7f1512000640 1 -- 172.21.15.26:0/2496531362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1514157f20 con 0x7f151414c9b0 2024-04-26T21:09:42.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.928+0000 7f1512000640 1 --2- 172.21.15.26:0/2496531362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f151414c9b0 0x7f151415a180 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f150002ab50 tx=0x7f1500004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:42.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.928+0000 7f150aa00640 1 -- 172.21.15.26:0/2496531362 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500040020 con 0x7f151414c9b0 2024-04-26T21:09:42.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.928+0000 7f150aa00640 1 -- 172.21.15.26:0/2496531362 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1500003b40 con 0x7f151414c9b0 2024-04-26T21:09:42.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.928+0000 7f150aa00640 1 -- 172.21.15.26:0/2496531362 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500036040 con 0x7f151414c9b0 2024-04-26T21:09:42.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.928+0000 7f1513e00640 1 -- 172.21.15.26:0/2496531362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15141592b0 con 0x7f151414c9b0 2024-04-26T21:09:42.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.928+0000 7f1513e00640 1 -- 172.21.15.26:0/2496531362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f151415d1e0 con 0x7f151414c9b0 2024-04-26T21:09:42.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.929+0000 7f1513e00640 1 -- 172.21.15.26:0/2496531362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f151406a700 con 0x7f151414c9b0 2024-04-26T21:09:42.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.930+0000 7f150aa00640 1 -- 172.21.15.26:0/2496531362 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1500043020 con 0x7f151414c9b0 2024-04-26T21:09:42.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.931+0000 7f150aa00640 1 --2- 172.21.15.26:0/2496531362 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f14f406c920 0x7f14f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:42.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.932+0000 7f150aa00640 1 -- 172.21.15.26:0/2496531362 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f15000ab3e0 con 0x7f151414c9b0 2024-04-26T21:09:42.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.932+0000 7f1512a00640 1 --2- 172.21.15.26:0/2496531362 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f14f406c920 0x7f14f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:42.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.933+0000 7f1512a00640 1 --2- 172.21.15.26:0/2496531362 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f14f406c920 0x7f14f406ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f15040099c0 tx=0x7f1504008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:42.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:42.936+0000 7f150aa00640 1 -- 172.21.15.26:0/2496531362 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f15000044e0 con 0x7f151414c9b0 2024-04-26T21:09:43.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.215+0000 7f1513e00640 1 -- 172.21.15.26:0/2496531362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f1514158430 con 0x7f151414c9b0 2024-04-26T21:09:43.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.220+0000 7f150aa00640 1 -- 172.21.15.26:0/2496531362 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f1500079bf0 con 0x7f151414c9b0 2024-04-26T21:09:43.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.224+0000 7f1513e00640 1 -- 172.21.15.26:0/2496531362 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f14f406c920 msgr2=0x7f14f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:43.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.224+0000 7f1513e00640 1 --2- 172.21.15.26:0/2496531362 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f14f406c920 0x7f14f406ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f15040099c0 tx=0x7f1504008040 comp rx=0 tx=0).stop 2024-04-26T21:09:43.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.224+0000 7f1513e00640 1 -- 172.21.15.26:0/2496531362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f151414c9b0 msgr2=0x7f151415a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:43.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.224+0000 7f1513e00640 1 --2- 172.21.15.26:0/2496531362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f151414c9b0 0x7f151415a180 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f150002ab50 tx=0x7f1500004730 comp rx=0 tx=0).stop 2024-04-26T21:09:43.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.224+0000 7f1513e00640 1 -- 172.21.15.26:0/2496531362 shutdown_connections 2024-04-26T21:09:43.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.224+0000 7f1513e00640 1 --2- 172.21.15.26:0/2496531362 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f14f406c920 0x7f14f406ede0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:43.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.224+0000 7f1513e00640 1 --2- 172.21.15.26:0/2496531362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15141577c0 0x7f151415a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:43.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.224+0000 7f1513e00640 1 --2- 172.21.15.26:0/2496531362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f151414c9b0 0x7f151415a180 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:43.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.224+0000 7f1513e00640 1 --2- 172.21.15.26:0/2496531362 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14f8004800 0x7f15140b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:43.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.224+0000 7f1513e00640 1 -- 172.21.15.26:0/2496531362 >> 172.21.15.26:0/2496531362 conn(0x7f151406c660 msgr2=0x7f151406ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:43.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.224+0000 7f1513e00640 1 -- 172.21.15.26:0/2496531362 shutdown_connections 2024-04-26T21:09:43.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.224+0000 7f1513e00640 1 -- 172.21.15.26:0/2496531362 wait complete. 2024-04-26T21:09:43.243 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-26T21:09:43.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.449+0000 7f11ae600640 1 Processor -- start 2024-04-26T21:09:43.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.449+0000 7f11ae600640 1 -- start start 2024-04-26T21:09:43.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.450+0000 7f11ae600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11a8146990 0x7f11a80b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:43.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.450+0000 7f11ae600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f11a806ec20 0x7f11a80d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:43.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.450+0000 7f11ae600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f11a80d1210 0x7f11a8072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:43.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.450+0000 7f11ae600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f11a8146d70 con 0x7f11a806ec20 2024-04-26T21:09:43.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.450+0000 7f11ae600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f11a80d1620 con 0x7f11a80d1210 2024-04-26T21:09:43.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.450+0000 7f11ae600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f11a80b4480 con 0x7f11a8146990 2024-04-26T21:09:43.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.451+0000 7f11a6a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f11a806ec20 0x7f11a80d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:43.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.451+0000 7f11a6a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f11a806ec20 0x7f11a80d0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47988/0 (socket says 172.21.15.26:47988) 2024-04-26T21:09:43.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.451+0000 7f11a6a00640 1 -- 172.21.15.26:0/2223213740 learned_addr learned my addr 172.21.15.26:0/2223213740 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:43.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.451+0000 7f11a7400640 1 --2- 172.21.15.26:0/2223213740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11a8146990 0x7f11a80b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:43.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.451+0000 7f11a7e00640 1 --2- 172.21.15.26:0/2223213740 >> v2:172.21.15.86:3300/0 conn(0x7f11a80d1210 0x7f11a8072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:43.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.451+0000 7f11a6a00640 1 -- 172.21.15.26:0/2223213740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11a8146990 msgr2=0x7f11a80b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:43.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.451+0000 7f11a6a00640 1 --2- 172.21.15.26:0/2223213740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11a8146990 0x7f11a80b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:43.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.451+0000 7f11a6a00640 1 -- 172.21.15.26:0/2223213740 >> v2:172.21.15.86:3300/0 conn(0x7f11a80d1210 msgr2=0x7f11a8072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:43.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.451+0000 7f11a6a00640 1 --2- 172.21.15.26:0/2223213740 >> v2:172.21.15.86:3300/0 conn(0x7f11a80d1210 0x7f11a8072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:43.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.451+0000 7f11a6a00640 1 -- 172.21.15.26:0/2223213740 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a8073b70 con 0x7f11a806ec20 2024-04-26T21:09:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.452+0000 7f11a6a00640 1 --2- 172.21.15.26:0/2223213740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11a806ec20 0x7f11a80d0cd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f11a815b0c0 tx=0x7f119002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf83d60bd2acfcb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.453+0000 7f11a6000640 1 -- 172.21.15.26:0/2223213740 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190037070 con 0x7f11a806ec20 2024-04-26T21:09:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.453+0000 7f11a6000640 1 -- 172.21.15.26:0/2223213740 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1190032440 con 0x7f11a806ec20 2024-04-26T21:09:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.453+0000 7f11a6000640 1 -- 172.21.15.26:0/2223213740 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190032810 con 0x7f11a806ec20 2024-04-26T21:09:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.453+0000 7f11ae600640 1 -- 172.21.15.26:0/2223213740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11a806ec20 msgr2=0x7f11a80d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:43.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.453+0000 7f11ae600640 1 --2- 172.21.15.26:0/2223213740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11a806ec20 0x7f11a80d0cd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f11a815b0c0 tx=0x7f119002a1e0 comp rx=0 tx=0).stop 2024-04-26T21:09:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.453+0000 7f11ae600640 1 -- 172.21.15.26:0/2223213740 shutdown_connections 2024-04-26T21:09:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.453+0000 7f11ae600640 1 --2- 172.21.15.26:0/2223213740 >> v2:172.21.15.86:3300/0 conn(0x7f11a80d1210 0x7f11a8072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.453+0000 7f11ae600640 1 --2- 172.21.15.26:0/2223213740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11a806ec20 0x7f11a80d0cd0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.453+0000 7f11ae600640 1 --2- 172.21.15.26:0/2223213740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11a8146990 0x7f11a80b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.453+0000 7f11ae600640 1 -- 172.21.15.26:0/2223213740 >> 172.21.15.26:0/2223213740 conn(0x7f11a80cd840 msgr2=0x7f11a80cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.453+0000 7f11ae600640 1 -- 172.21.15.26:0/2223213740 shutdown_connections 2024-04-26T21:09:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.453+0000 7f11ae600640 1 -- 172.21.15.26:0/2223213740 wait complete. 2024-04-26T21:09:43.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.455+0000 7f11ae600640 1 Processor -- start 2024-04-26T21:09:43.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.455+0000 7f11ae600640 1 -- start start 2024-04-26T21:09:43.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.456+0000 7f11ae600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11a806ec20 0x7f11a80af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:43.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.456+0000 7f11ae600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a80d1210 0x7f11a80b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:43.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.456+0000 7f11ae600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11a8146990 0x7f11a8155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:43.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.456+0000 7f11ae600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f11a806e420 con 0x7f11a8146990 2024-04-26T21:09:43.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.456+0000 7f11ae600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11a8074130 con 0x7f11a80d1210 2024-04-26T21:09:43.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.456+0000 7f11ae600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f11a806e700 con 0x7f11a806ec20 2024-04-26T21:09:43.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.456+0000 7f11a7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11a8146990 0x7f11a8155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:43.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.456+0000 7f11a7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11a8146990 0x7f11a8155460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48004/0 (socket says 172.21.15.26:48004) 2024-04-26T21:09:43.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.456+0000 7f11a7e00640 1 -- 172.21.15.26:0/2167812348 learned_addr learned my addr 172.21.15.26:0/2167812348 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:43.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.457+0000 7f11a7e00640 1 -- 172.21.15.26:0/2167812348 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11a806ec20 msgr2=0x7f11a80af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:43.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.457+0000 7f11a7e00640 1 --2- 172.21.15.26:0/2167812348 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11a806ec20 0x7f11a80af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:43.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.457+0000 7f11a7e00640 1 -- 172.21.15.26:0/2167812348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a80d1210 msgr2=0x7f11a80b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:43.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.457+0000 7f11a7e00640 1 --2- 172.21.15.26:0/2167812348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a80d1210 0x7f11a80b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:43.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.457+0000 7f11a7e00640 1 -- 172.21.15.26:0/2167812348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a80cea80 con 0x7f11a8146990 2024-04-26T21:09:43.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.457+0000 7f11a7e00640 1 --2- 172.21.15.26:0/2167812348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11a8146990 0x7f11a8155460 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f11a000d5d0 tx=0x7f11a000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:43.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.457+0000 7f119f400640 1 -- 172.21.15.26:0/2167812348 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a00125f0 con 0x7f11a8146990 2024-04-26T21:09:43.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.457+0000 7f119f400640 1 -- 172.21.15.26:0/2167812348 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11a0019070 con 0x7f11a8146990 2024-04-26T21:09:43.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.457+0000 7f119f400640 1 -- 172.21.15.26:0/2167812348 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a00129f0 con 0x7f11a8146990 2024-04-26T21:09:43.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.457+0000 7f11ae600640 1 -- 172.21.15.26:0/2167812348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a80ced60 con 0x7f11a8146990 2024-04-26T21:09:43.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.458+0000 7f11ae600640 1 -- 172.21.15.26:0/2167812348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a80cf750 con 0x7f11a8146990 2024-04-26T21:09:43.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.459+0000 7f11ae600640 1 -- 172.21.15.26:0/2167812348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a806a700 con 0x7f11a8146990 2024-04-26T21:09:43.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.460+0000 7f119f400640 1 -- 172.21.15.26:0/2167812348 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f11a0012b90 con 0x7f11a8146990 2024-04-26T21:09:43.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.461+0000 7f119f400640 1 --2- 172.21.15.26:0/2167812348 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f118406c8f0 0x7f118406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:43.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.461+0000 7f119f400640 1 -- 172.21.15.26:0/2167812348 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f11a008bc80 con 0x7f11a8146990 2024-04-26T21:09:43.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.461+0000 7f11a7400640 1 --2- 172.21.15.26:0/2167812348 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f118406c8f0 0x7f118406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:43.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.462+0000 7f11a7400640 1 --2- 172.21.15.26:0/2167812348 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f118406c8f0 0x7f118406edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1198009540 tx=0x7f1198009450 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:43.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.466+0000 7f119f400640 1 -- 172.21.15.26:0/2167812348 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f11a005a370 con 0x7f11a8146990 2024-04-26T21:09:43.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.742+0000 7f11ae600640 1 -- 172.21.15.26:0/2167812348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f11a806fd50 con 0x7f11a8146990 2024-04-26T21:09:43.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.747+0000 7f119f400640 1 -- 172.21.15.26:0/2167812348 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f11a0060070 con 0x7f11a8146990 2024-04-26T21:09:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.749+0000 7f11ae600640 1 -- 172.21.15.26:0/2167812348 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f118406c8f0 msgr2=0x7f118406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.749+0000 7f11ae600640 1 --2- 172.21.15.26:0/2167812348 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f118406c8f0 0x7f118406edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1198009540 tx=0x7f1198009450 comp rx=0 tx=0).stop 2024-04-26T21:09:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.749+0000 7f11ae600640 1 -- 172.21.15.26:0/2167812348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11a8146990 msgr2=0x7f11a8155460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:43.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.749+0000 7f11ae600640 1 --2- 172.21.15.26:0/2167812348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11a8146990 0x7f11a8155460 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f11a000d5d0 tx=0x7f11a000da90 comp rx=0 tx=0).stop 2024-04-26T21:09:43.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.749+0000 7f11ae600640 1 -- 172.21.15.26:0/2167812348 shutdown_connections 2024-04-26T21:09:43.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.749+0000 7f11ae600640 1 --2- 172.21.15.26:0/2167812348 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f118406c8f0 0x7f118406edb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:43.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.749+0000 7f11ae600640 1 --2- 172.21.15.26:0/2167812348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11a8146990 0x7f11a8155460 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:43.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.749+0000 7f11ae600640 1 --2- 172.21.15.26:0/2167812348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a80d1210 0x7f11a80b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:43.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.750+0000 7f11ae600640 1 --2- 172.21.15.26:0/2167812348 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11a806ec20 0x7f11a80af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:43.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.750+0000 7f11ae600640 1 -- 172.21.15.26:0/2167812348 >> 172.21.15.26:0/2167812348 conn(0x7f11a80cd840 msgr2=0x7f11a80b6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:43.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.750+0000 7f11ae600640 1 -- 172.21.15.26:0/2167812348 shutdown_connections 2024-04-26T21:09:43.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.750+0000 7f11ae600640 1 -- 172.21.15.26:0/2167812348 wait complete. 2024-04-26T21:09:43.770 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-26T21:09:43.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.972+0000 7f347ac00640 1 Processor -- start 2024-04-26T21:09:43.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.972+0000 7f347ac00640 1 -- start start 2024-04-26T21:09:43.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.974+0000 7f347ac00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3474162bf0 0x7f34740d5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:43.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.974+0000 7f347ac00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34740b7180 0x7f34740b8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:43.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.974+0000 7f347ac00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34740d3750 con 0x7f3474162810 2024-04-26T21:09:43.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.974+0000 7f347ac00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f34740d3a30 con 0x7f34740b7180 2024-04-26T21:09:43.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.974+0000 7f347ac00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f34740d3d10 con 0x7f3474162bf0 2024-04-26T21:09:43.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.974+0000 7f3479800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3474162810 0x7f34740b6a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32922/0 (socket says 172.21.15.26:32922) 2024-04-26T21:09:43.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.974+0000 7f3479800640 1 -- 172.21.15.26:0/110749218 learned_addr learned my addr 172.21.15.26:0/110749218 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:43.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.975+0000 7f347a200640 1 --2- 172.21.15.26:0/110749218 >> v2:172.21.15.26:3300/0 conn(0x7f34740b7180 0x7f34740b8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:43.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.975+0000 7f3478e00640 1 --2- 172.21.15.26:0/110749218 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3474162bf0 0x7f34740d5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:43.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.975+0000 7f347a200640 1 -- 172.21.15.26:0/110749218 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3474162bf0 msgr2=0x7f34740d5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:43.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.975+0000 7f347a200640 1 --2- 172.21.15.26:0/110749218 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3474162bf0 0x7f34740d5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:43.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.975+0000 7f347a200640 1 -- 172.21.15.26:0/110749218 >> v1:172.21.15.26:6789/0 conn(0x7f3474162810 legacy=0x7f34740b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:43.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.975+0000 7f3473e00640 1 -- 172.21.15.26:0/110749218 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3364493077 0 0) 0x7f34740d3750 con 0x7f3474162810 2024-04-26T21:09:43.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.975+0000 7f347a200640 1 -- 172.21.15.26:0/110749218 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34740d0e30 con 0x7f34740b7180 2024-04-26T21:09:43.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.976+0000 7f347a200640 1 --2- 172.21.15.26:0/110749218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34740b7180 0x7f34740b8220 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f346c0097a0 tx=0x7f346c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3d09f0b1be50f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:43.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.977+0000 7f3473e00640 1 -- 172.21.15.26:0/110749218 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c037070 con 0x7f34740b7180 2024-04-26T21:09:43.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.977+0000 7f3473e00640 1 -- 172.21.15.26:0/110749218 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f346c032440 con 0x7f34740b7180 2024-04-26T21:09:43.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.977+0000 7f3473e00640 1 -- 172.21.15.26:0/110749218 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c032810 con 0x7f34740b7180 2024-04-26T21:09:43.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.977+0000 7f347ac00640 1 -- 172.21.15.26:0/110749218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34740b7180 msgr2=0x7f34740b8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:43.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.977+0000 7f347ac00640 1 --2- 172.21.15.26:0/110749218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34740b7180 0x7f34740b8220 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f346c0097a0 tx=0x7f346c02a180 comp rx=0 tx=0).stop 2024-04-26T21:09:43.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.977+0000 7f347ac00640 1 -- 172.21.15.26:0/110749218 shutdown_connections 2024-04-26T21:09:43.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.977+0000 7f347ac00640 1 --2- 172.21.15.26:0/110749218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34740b7180 0x7f34740b8220 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:43.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.977+0000 7f347ac00640 1 --2- 172.21.15.26:0/110749218 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3474162bf0 0x7f34740d5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:43.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.977+0000 7f347ac00640 1 -- 172.21.15.26:0/110749218 >> 172.21.15.26:0/110749218 conn(0x7f347409e380 msgr2=0x7f34740d4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:43.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.977+0000 7f347ac00640 1 -- 172.21.15.26:0/110749218 shutdown_connections 2024-04-26T21:09:43.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.977+0000 7f347ac00640 1 -- 172.21.15.26:0/110749218 wait complete. 2024-04-26T21:09:43.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.979+0000 7f347ac00640 1 Processor -- start 2024-04-26T21:09:43.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.979+0000 7f347ac00640 1 -- start start 2024-04-26T21:09:43.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.980+0000 7f347ac00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34740b7180 0x7f347415e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:43.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.980+0000 7f347ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3474162810 0x7f34741f4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:43.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.980+0000 7f347ac00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3474162bf0 0x7f347415fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:43.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.980+0000 7f347ac00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f34740b8760 con 0x7f34740b7180 2024-04-26T21:09:43.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.980+0000 7f347ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34740b6150 con 0x7f3474162810 2024-04-26T21:09:43.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.980+0000 7f347ac00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f34740b88e0 con 0x7f3474162bf0 2024-04-26T21:09:43.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.980+0000 7f3479800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34740b7180 0x7f347415e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:43.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.980+0000 7f3479800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34740b7180 0x7f347415e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48022/0 (socket says 172.21.15.26:48022) 2024-04-26T21:09:43.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.981+0000 7f3479800640 1 -- 172.21.15.26:0/1609088191 learned_addr learned my addr 172.21.15.26:0/1609088191 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.981+0000 7f347a200640 1 --2- 172.21.15.26:0/1609088191 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3474162bf0 0x7f347415fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.981+0000 7f3479800640 1 -- 172.21.15.26:0/1609088191 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3474162bf0 msgr2=0x7f347415fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.981+0000 7f3478e00640 1 --2- 172.21.15.26:0/1609088191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3474162810 0x7f34741f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.981+0000 7f3479800640 1 --2- 172.21.15.26:0/1609088191 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3474162bf0 0x7f347415fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.981+0000 7f3479800640 1 -- 172.21.15.26:0/1609088191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3474162810 msgr2=0x7f34741f4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.981+0000 7f3479800640 1 --2- 172.21.15.26:0/1609088191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3474162810 0x7f34741f4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.981+0000 7f3479800640 1 -- 172.21.15.26:0/1609088191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3474156fe0 con 0x7f34740b7180 2024-04-26T21:09:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.981+0000 7f3479800640 1 --2- 172.21.15.26:0/1609088191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34740b7180 0x7f347415e740 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f346800c890 tx=0x7f346800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.982+0000 7f3471600640 1 -- 172.21.15.26:0/1609088191 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3468018070 con 0x7f34740b7180 2024-04-26T21:09:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.982+0000 7f3471600640 1 -- 172.21.15.26:0/1609088191 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f346800a640 con 0x7f34740b7180 2024-04-26T21:09:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.982+0000 7f3471600640 1 -- 172.21.15.26:0/1609088191 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346800aa30 con 0x7f34740b7180 2024-04-26T21:09:43.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.982+0000 7f347ac00640 1 -- 172.21.15.26:0/1609088191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347415a950 con 0x7f34740b7180 2024-04-26T21:09:43.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.982+0000 7f347ac00640 1 -- 172.21.15.26:0/1609088191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f347415f0b0 con 0x7f34740b7180 2024-04-26T21:09:43.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.983+0000 7f347ac00640 1 -- 172.21.15.26:0/1609088191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f347406a700 con 0x7f34740b7180 2024-04-26T21:09:43.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.984+0000 7f3471600640 1 -- 172.21.15.26:0/1609088191 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f346801b070 con 0x7f34740b7180 2024-04-26T21:09:43.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.985+0000 7f3471600640 1 --2- 172.21.15.26:0/1609088191 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f345406c8f0 0x7f345406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:43.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.985+0000 7f3471600640 1 -- 172.21.15.26:0/1609088191 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f346808af10 con 0x7f34740b7180 2024-04-26T21:09:43.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.985+0000 7f3478e00640 1 --2- 172.21.15.26:0/1609088191 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f345406c8f0 0x7f345406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:43.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.986+0000 7f3478e00640 1 --2- 172.21.15.26:0/1609088191 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f345406c8f0 0x7f345406edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3460006fd0 tx=0x7f3460008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:43.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:43.989+0000 7f3471600640 1 -- 172.21.15.26:0/1609088191 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f34680595f0 con 0x7f34740b7180 2024-04-26T21:09:44.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.273+0000 7f347ac00640 1 -- 172.21.15.26:0/1609088191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f347415f2f0 con 0x7f34740b7180 2024-04-26T21:09:44.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.277+0000 7f3471600640 1 -- 172.21.15.26:0/1609088191 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f3468058ef0 con 0x7f34740b7180 2024-04-26T21:09:44.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.280+0000 7f347ac00640 1 -- 172.21.15.26:0/1609088191 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f345406c8f0 msgr2=0x7f345406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:44.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.280+0000 7f347ac00640 1 --2- 172.21.15.26:0/1609088191 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f345406c8f0 0x7f345406edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3460006fd0 tx=0x7f3460008040 comp rx=0 tx=0).stop 2024-04-26T21:09:44.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.280+0000 7f347ac00640 1 -- 172.21.15.26:0/1609088191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34740b7180 msgr2=0x7f347415e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:44.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.280+0000 7f347ac00640 1 --2- 172.21.15.26:0/1609088191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34740b7180 0x7f347415e740 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f346800c890 tx=0x7f346800cd50 comp rx=0 tx=0).stop 2024-04-26T21:09:44.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.280+0000 7f347ac00640 1 -- 172.21.15.26:0/1609088191 shutdown_connections 2024-04-26T21:09:44.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.280+0000 7f347ac00640 1 --2- 172.21.15.26:0/1609088191 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f345406c8f0 0x7f345406edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:44.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.280+0000 7f347ac00640 1 --2- 172.21.15.26:0/1609088191 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3474162bf0 0x7f347415fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:44.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.280+0000 7f347ac00640 1 --2- 172.21.15.26:0/1609088191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3474162810 0x7f34741f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:44.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.280+0000 7f347ac00640 1 --2- 172.21.15.26:0/1609088191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34740b7180 0x7f347415e740 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:44.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.280+0000 7f347ac00640 1 -- 172.21.15.26:0/1609088191 >> 172.21.15.26:0/1609088191 conn(0x7f347409e380 msgr2=0x7f3474161800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:44.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.280+0000 7f347ac00640 1 -- 172.21.15.26:0/1609088191 shutdown_connections 2024-04-26T21:09:44.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.281+0000 7f347ac00640 1 -- 172.21.15.26:0/1609088191 wait complete. 2024-04-26T21:09:44.303 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-26T21:09:44.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.513+0000 7f4e3fe00640 1 Processor -- start 2024-04-26T21:09:44.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.514+0000 7f4e3fe00640 1 -- start start 2024-04-26T21:09:44.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.515+0000 7f4e3fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4e40146830 0x7f4e4015d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:44.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.516+0000 7f4e3fe00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e400b4280 con 0x7f4e400b3d10 2024-04-26T21:09:44.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.516+0000 7f4e3fe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e400b4560 con 0x7f4e400b2e30 2024-04-26T21:09:44.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.516+0000 7f4e3fe00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f4e40146c10 con 0x7f4e40146830 2024-04-26T21:09:44.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.516+0000 7f4e3ea00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4e40146830 0x7f4e4015d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:44.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.516+0000 7f4e3e000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4e400b3d10 0x7f4e40073f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32930/0 (socket says 172.21.15.26:32930) 2024-04-26T21:09:44.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.516+0000 7f4e3e000640 1 -- 172.21.15.26:0/3983204174 learned_addr learned my addr 172.21.15.26:0/3983204174 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:44.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.517+0000 7f4e3ea00640 1 -- 172.21.15.26:0/3983204174 >> v1:172.21.15.86:6789/0 conn(0x7f4e400b2e30 legacy=0x7f4e400b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:44.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.517+0000 7f4e3ea00640 1 -- 172.21.15.26:0/3983204174 >> v1:172.21.15.26:6789/0 conn(0x7f4e400b3d10 legacy=0x7f4e40073f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:44.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.517+0000 7f4e3ea00640 1 -- 172.21.15.26:0/3983204174 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e400b4840 con 0x7f4e40146830 2024-04-26T21:09:44.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.518+0000 7f4e3ea00640 1 --2- 172.21.15.26:0/3983204174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e40146830 0x7f4e4015d6b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4e300097d0 tx=0x7f4e3002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5ed834080f7482 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:44.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.518+0000 7f4e3d600640 1 -- 172.21.15.26:0/3983204174 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30037070 con 0x7f4e40146830 2024-04-26T21:09:44.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.518+0000 7f4e3d600640 1 -- 172.21.15.26:0/3983204174 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e30032440 con 0x7f4e40146830 2024-04-26T21:09:44.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.518+0000 7f4e3d600640 1 -- 172.21.15.26:0/3983204174 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30032810 con 0x7f4e40146830 2024-04-26T21:09:44.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.519+0000 7f4e3fe00640 1 -- 172.21.15.26:0/3983204174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e40146830 msgr2=0x7f4e4015d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:44.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.519+0000 7f4e3fe00640 1 --2- 172.21.15.26:0/3983204174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e40146830 0x7f4e4015d6b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4e300097d0 tx=0x7f4e3002a200 comp rx=0 tx=0).stop 2024-04-26T21:09:44.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.519+0000 7f4e3fe00640 1 -- 172.21.15.26:0/3983204174 shutdown_connections 2024-04-26T21:09:44.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.519+0000 7f4e3fe00640 1 --2- 172.21.15.26:0/3983204174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e40146830 0x7f4e4015d6b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:44.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.519+0000 7f4e3fe00640 1 -- 172.21.15.26:0/3983204174 >> 172.21.15.26:0/3983204174 conn(0x7f4e4006c660 msgr2=0x7f4e4006ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:44.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.519+0000 7f4e3fe00640 1 -- 172.21.15.26:0/3983204174 shutdown_connections 2024-04-26T21:09:44.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.519+0000 7f4e3fe00640 1 -- 172.21.15.26:0/3983204174 wait complete. 2024-04-26T21:09:44.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.521+0000 7f4e3fe00640 1 Processor -- start 2024-04-26T21:09:44.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.521+0000 7f4e3fe00640 1 -- start start 2024-04-26T21:09:44.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.523+0000 7f4e3fe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e400b2e30 0x7f4e4014eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:44.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.523+0000 7f4e3fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e400b3d10 0x7f4e4013d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:44.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.523+0000 7f4e3fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e40146830 0x7f4e4014d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:44.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.523+0000 7f4e3fe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4e400b32c0 con 0x7f4e400b3d10 2024-04-26T21:09:44.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.523+0000 7f4e3fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e400b4e00 con 0x7f4e40146830 2024-04-26T21:09:44.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.523+0000 7f4e3fe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f4e400b3440 con 0x7f4e400b2e30 2024-04-26T21:09:44.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.523+0000 7f4e3e000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e400b3d10 0x7f4e4013d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:44.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.523+0000 7f4e3e000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e400b3d10 0x7f4e4013d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48040/0 (socket says 172.21.15.26:48040) 2024-04-26T21:09:44.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.523+0000 7f4e3e000640 1 -- 172.21.15.26:0/1996711742 learned_addr learned my addr 172.21.15.26:0/1996711742 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:44.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.523+0000 7f4e3ea00640 1 --2- 172.21.15.26:0/1996711742 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e400b2e30 0x7f4e4014eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:44.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.523+0000 7f4e3f400640 1 --2- 172.21.15.26:0/1996711742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e40146830 0x7f4e4014d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:44.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.523+0000 7f4e3e000640 1 -- 172.21.15.26:0/1996711742 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e400b2e30 msgr2=0x7f4e4014eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:44.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.523+0000 7f4e3e000640 1 --2- 172.21.15.26:0/1996711742 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e400b2e30 0x7f4e4014eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:44.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.523+0000 7f4e3e000640 1 -- 172.21.15.26:0/1996711742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e40146830 msgr2=0x7f4e4014d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:44.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.523+0000 7f4e3e000640 1 --2- 172.21.15.26:0/1996711742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e40146830 0x7f4e4014d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:44.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.523+0000 7f4e3e000640 1 -- 172.21.15.26:0/1996711742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4014dd00 con 0x7f4e400b3d10 2024-04-26T21:09:44.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.524+0000 7f4e3e000640 1 --2- 172.21.15.26:0/1996711742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e400b3d10 0x7f4e4013d830 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00c400 tx=0x7f4e2c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:44.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.524+0000 7f4e36a00640 1 -- 172.21.15.26:0/1996711742 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c017070 con 0x7f4e400b3d10 2024-04-26T21:09:44.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.524+0000 7f4e36a00640 1 -- 172.21.15.26:0/1996711742 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e2c005ce0 con 0x7f4e400b3d10 2024-04-26T21:09:44.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.524+0000 7f4e3fe00640 1 -- 172.21.15.26:0/1996711742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e4014dfe0 con 0x7f4e400b3d10 2024-04-26T21:09:44.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.524+0000 7f4e36a00640 1 -- 172.21.15.26:0/1996711742 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c0072f0 con 0x7f4e400b3d10 2024-04-26T21:09:44.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.524+0000 7f4e3fe00640 1 -- 172.21.15.26:0/1996711742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e4014e5a0 con 0x7f4e400b3d10 2024-04-26T21:09:44.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.525+0000 7f4e3fe00640 1 -- 172.21.15.26:0/1996711742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e4006a700 con 0x7f4e400b3d10 2024-04-26T21:09:44.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.527+0000 7f4e36a00640 1 -- 172.21.15.26:0/1996711742 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e2c01a070 con 0x7f4e400b3d10 2024-04-26T21:09:44.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.528+0000 7f4e36a00640 1 --2- 172.21.15.26:0/1996711742 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4e2006c920 0x7f4e2006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:44.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.528+0000 7f4e36a00640 1 -- 172.21.15.26:0/1996711742 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f4e2c08ac60 con 0x7f4e400b3d10 2024-04-26T21:09:44.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.528+0000 7f4e3ea00640 1 --2- 172.21.15.26:0/1996711742 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4e2006c920 0x7f4e2006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:44.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.529+0000 7f4e3ea00640 1 --2- 172.21.15.26:0/1996711742 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4e2006c920 0x7f4e2006ede0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4e30034000 tx=0x7f4e3002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:44.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.532+0000 7f4e36a00640 1 -- 172.21.15.26:0/1996711742 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4e2c059340 con 0x7f4e400b3d10 2024-04-26T21:09:44.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.810+0000 7f4e3fe00640 1 -- 172.21.15.26:0/1996711742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f4e400d0b00 con 0x7f4e400b3d10 2024-04-26T21:09:44.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.815+0000 7f4e36a00640 1 -- 172.21.15.26:0/1996711742 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f4e2c058c40 con 0x7f4e400b3d10 2024-04-26T21:09:44.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.817+0000 7f4e3fe00640 1 -- 172.21.15.26:0/1996711742 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4e2006c920 msgr2=0x7f4e2006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:44.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.817+0000 7f4e3fe00640 1 --2- 172.21.15.26:0/1996711742 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4e2006c920 0x7f4e2006ede0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4e30034000 tx=0x7f4e3002e000 comp rx=0 tx=0).stop 2024-04-26T21:09:44.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.817+0000 7f4e3fe00640 1 -- 172.21.15.26:0/1996711742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e400b3d10 msgr2=0x7f4e4013d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:44.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.817+0000 7f4e3fe00640 1 --2- 172.21.15.26:0/1996711742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e400b3d10 0x7f4e4013d830 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00c400 tx=0x7f4e2c00c8c0 comp rx=0 tx=0).stop 2024-04-26T21:09:44.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.817+0000 7f4e3fe00640 1 -- 172.21.15.26:0/1996711742 shutdown_connections 2024-04-26T21:09:44.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.817+0000 7f4e3fe00640 1 --2- 172.21.15.26:0/1996711742 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f4e2006c920 0x7f4e2006ede0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:44.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.817+0000 7f4e3fe00640 1 --2- 172.21.15.26:0/1996711742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e40146830 0x7f4e4014d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:44.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.817+0000 7f4e3fe00640 1 --2- 172.21.15.26:0/1996711742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e400b3d10 0x7f4e4013d830 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:44.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.817+0000 7f4e3fe00640 1 --2- 172.21.15.26:0/1996711742 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e400b2e30 0x7f4e4014eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:44.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.817+0000 7f4e3fe00640 1 -- 172.21.15.26:0/1996711742 >> 172.21.15.26:0/1996711742 conn(0x7f4e4006c660 msgr2=0x7f4e4015f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:44.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.817+0000 7f4e3fe00640 1 -- 172.21.15.26:0/1996711742 shutdown_connections 2024-04-26T21:09:44.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:44.818+0000 7f4e3fe00640 1 -- 172.21.15.26:0/1996711742 wait complete. 2024-04-26T21:09:44.838 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:09:45.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.042+0000 7f05fe400640 1 Processor -- start 2024-04-26T21:09:45.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.042+0000 7f05fe400640 1 -- start start 2024-04-26T21:09:45.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.043+0000 7f05fe400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f05f8155c10 0x7f05f80b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.044+0000 7f05fe400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f05f80d0670 0x7f05f80728f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.044+0000 7f05fe400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05f815c8c0 con 0x7f05f8071fe0 2024-04-26T21:09:45.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.044+0000 7f05fe400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f05f80b4f30 con 0x7f05f8155c10 2024-04-26T21:09:45.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.044+0000 7f05fe400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f05f80b5210 con 0x7f05f80d0670 2024-04-26T21:09:45.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.044+0000 7f05fd000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f05f8155c10 0x7f05f80b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.044+0000 7f05fd000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f05f8155c10 0x7f05f80b5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48046/0 (socket says 172.21.15.26:48046) 2024-04-26T21:09:45.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.044+0000 7f05fd000640 1 -- 172.21.15.26:0/2278952061 learned_addr learned my addr 172.21.15.26:0/2278952061 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:45.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.044+0000 7f05fda00640 1 --2- 172.21.15.26:0/2278952061 >> v2:172.21.15.86:3300/0 conn(0x7f05f80d0670 0x7f05f80728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.044+0000 7f05fd000640 1 -- 172.21.15.26:0/2278952061 >> v2:172.21.15.86:3300/0 conn(0x7f05f80d0670 msgr2=0x7f05f80728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:45.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.044+0000 7f05fd000640 1 --2- 172.21.15.26:0/2278952061 >> v2:172.21.15.86:3300/0 conn(0x7f05f80d0670 0x7f05f80728f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.044+0000 7f05fd000640 1 -- 172.21.15.26:0/2278952061 >> v1:172.21.15.86:6789/0 conn(0x7f05f8071fe0 legacy=0x7f05f806e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:45.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.044+0000 7f05fd000640 1 -- 172.21.15.26:0/2278952061 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f80b54f0 con 0x7f05f8155c10 2024-04-26T21:09:45.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.046+0000 7f05fd000640 1 --2- 172.21.15.26:0/2278952061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05f8155c10 0x7f05f80b5a10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f05f815f1f0 tx=0x7f05e802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5836b2357d33c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:45.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.046+0000 7f05f7400640 1 -- 172.21.15.26:0/2278952061 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8037070 con 0x7f05f8155c10 2024-04-26T21:09:45.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.046+0000 7f05f7400640 1 -- 172.21.15.26:0/2278952061 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05e8032440 con 0x7f05f8155c10 2024-04-26T21:09:45.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.046+0000 7f05f7400640 1 -- 172.21.15.26:0/2278952061 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8032810 con 0x7f05f8155c10 2024-04-26T21:09:45.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.047+0000 7f05fe400640 1 -- 172.21.15.26:0/2278952061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05f8155c10 msgr2=0x7f05f80b5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:45.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.047+0000 7f05fe400640 1 --2- 172.21.15.26:0/2278952061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05f8155c10 0x7f05f80b5a10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f05f815f1f0 tx=0x7f05e802a230 comp rx=0 tx=0).stop 2024-04-26T21:09:45.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.047+0000 7f05fe400640 1 -- 172.21.15.26:0/2278952061 shutdown_connections 2024-04-26T21:09:45.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.047+0000 7f05fe400640 1 --2- 172.21.15.26:0/2278952061 >> v2:172.21.15.86:3300/0 conn(0x7f05f80d0670 0x7f05f80728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.047+0000 7f05fe400640 1 --2- 172.21.15.26:0/2278952061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05f8155c10 0x7f05f80b5a10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.047+0000 7f05fe400640 1 -- 172.21.15.26:0/2278952061 >> 172.21.15.26:0/2278952061 conn(0x7f05f806c690 msgr2=0x7f05f806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:45.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.047+0000 7f05fe400640 1 -- 172.21.15.26:0/2278952061 shutdown_connections 2024-04-26T21:09:45.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.047+0000 7f05fe400640 1 -- 172.21.15.26:0/2278952061 wait complete. 2024-04-26T21:09:45.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.049+0000 7f05fe400640 1 Processor -- start 2024-04-26T21:09:45.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.049+0000 7f05fe400640 1 -- start start 2024-04-26T21:09:45.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.050+0000 7f05fe400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05f8071fe0 0x7f05f81f9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.050+0000 7f05fe400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05f80d0670 0x7f05f81500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.050+0000 7f05fe400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05f8155c10 0x7f05f806f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.050+0000 7f05fe400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f05f8156170 con 0x7f05f80d0670 2024-04-26T21:09:45.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.050+0000 7f05fe400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05f8155ff0 con 0x7f05f8155c10 2024-04-26T21:09:45.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.050+0000 7f05fe400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f05f80735d0 con 0x7f05f8071fe0 2024-04-26T21:09:45.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.050+0000 7f05f7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05f80d0670 0x7f05f81500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.050+0000 7f05f7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05f80d0670 0x7f05f81500d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48048/0 (socket says 172.21.15.26:48048) 2024-04-26T21:09:45.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.050+0000 7f05f7e00640 1 -- 172.21.15.26:0/2259419200 learned_addr learned my addr 172.21.15.26:0/2259419200 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:45.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.051+0000 7f05fda00640 1 --2- 172.21.15.26:0/2259419200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05f8155c10 0x7f05f806f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.051+0000 7f05fd000640 1 --2- 172.21.15.26:0/2259419200 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05f8071fe0 0x7f05f81f9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.051+0000 7f05f7e00640 1 -- 172.21.15.26:0/2259419200 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05f8071fe0 msgr2=0x7f05f81f9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:45.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.051+0000 7f05f7e00640 1 --2- 172.21.15.26:0/2259419200 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05f8071fe0 0x7f05f81f9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.051+0000 7f05f7e00640 1 -- 172.21.15.26:0/2259419200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05f8155c10 msgr2=0x7f05f806f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:45.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.051+0000 7f05f7e00640 1 --2- 172.21.15.26:0/2259419200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05f8155c10 0x7f05f806f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.051+0000 7f05f7e00640 1 -- 172.21.15.26:0/2259419200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f81f9960 con 0x7f05f80d0670 2024-04-26T21:09:45.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.051+0000 7f05fda00640 1 --2- 172.21.15.26:0/2259419200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05f8155c10 0x7f05f806f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:45.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.051+0000 7f05fd000640 1 --2- 172.21.15.26:0/2259419200 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05f8071fe0 0x7f05f81f9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:45.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.051+0000 7f05f7e00640 1 --2- 172.21.15.26:0/2259419200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05f80d0670 0x7f05f81500d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f05e400b3e0 tx=0x7f05e400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:45.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.051+0000 7f05f4c00640 1 -- 172.21.15.26:0/2259419200 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e4015020 con 0x7f05f80d0670 2024-04-26T21:09:45.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.051+0000 7f05f4c00640 1 -- 172.21.15.26:0/2259419200 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05e4011850 con 0x7f05f80d0670 2024-04-26T21:09:45.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.051+0000 7f05f4c00640 1 -- 172.21.15.26:0/2259419200 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e4011c40 con 0x7f05f80d0670 2024-04-26T21:09:45.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.051+0000 7f05fe400640 1 -- 172.21.15.26:0/2259419200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f80cdee0 con 0x7f05f80d0670 2024-04-26T21:09:45.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.052+0000 7f05fe400640 1 -- 172.21.15.26:0/2259419200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f80ce4a0 con 0x7f05f80d0670 2024-04-26T21:09:45.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.053+0000 7f05fe400640 1 -- 172.21.15.26:0/2259419200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f80cec70 con 0x7f05f80d0670 2024-04-26T21:09:45.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.054+0000 7f05f4c00640 1 -- 172.21.15.26:0/2259419200 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f05e4018020 con 0x7f05f80d0670 2024-04-26T21:09:45.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.055+0000 7f05f4c00640 1 --2- 172.21.15.26:0/2259419200 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05d806c8f0 0x7f05d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.055+0000 7f05f4c00640 1 -- 172.21.15.26:0/2259419200 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f05e408b810 con 0x7f05f80d0670 2024-04-26T21:09:45.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.055+0000 7f05fd000640 1 --2- 172.21.15.26:0/2259419200 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05d806c8f0 0x7f05d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.056+0000 7f05fd000640 1 --2- 172.21.15.26:0/2259419200 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05d806c8f0 0x7f05d806edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f05e8034000 tx=0x7f05e802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:45.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.060+0000 7f05f4c00640 1 -- 172.21.15.26:0/2259419200 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f05e401f020 con 0x7f05f80d0670 2024-04-26T21:09:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.271+0000 7f05fe400640 1 -- 172.21.15.26:0/2259419200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05f806a700 con 0x7f05f80d0670 2024-04-26T21:09:45.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.273+0000 7f05f4c00640 1 -- 172.21.15.26:0/2259419200 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+12843 (secure 0 0 0) 0x7f05f81f9960 con 0x7f05f80d0670 2024-04-26T21:09:45.274 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:45.274 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:09:41.113055+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:09:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.276+0000 7f05fe400640 1 -- 172.21.15.26:0/2259419200 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05d806c8f0 msgr2=0x7f05d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.276+0000 7f05fe400640 1 --2- 172.21.15.26:0/2259419200 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05d806c8f0 0x7f05d806edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f05e8034000 tx=0x7f05e802e000 comp rx=0 tx=0).stop 2024-04-26T21:09:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.276+0000 7f05fe400640 1 -- 172.21.15.26:0/2259419200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05f80d0670 msgr2=0x7f05f81500d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.276+0000 7f05fe400640 1 --2- 172.21.15.26:0/2259419200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05f80d0670 0x7f05f81500d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f05e400b3e0 tx=0x7f05e400b8a0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.276+0000 7f05fe400640 1 -- 172.21.15.26:0/2259419200 shutdown_connections 2024-04-26T21:09:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.276+0000 7f05fe400640 1 --2- 172.21.15.26:0/2259419200 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05d806c8f0 0x7f05d806edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.276+0000 7f05fe400640 1 --2- 172.21.15.26:0/2259419200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05f8155c10 0x7f05f806f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.276+0000 7f05fe400640 1 --2- 172.21.15.26:0/2259419200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05f80d0670 0x7f05f81500d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.276+0000 7f05fe400640 1 --2- 172.21.15.26:0/2259419200 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05f8071fe0 0x7f05f81f9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.276+0000 7f05fe400640 1 -- 172.21.15.26:0/2259419200 >> 172.21.15.26:0/2259419200 conn(0x7f05f806c690 msgr2=0x7f05f80b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.276+0000 7f05fe400640 1 -- 172.21.15.26:0/2259419200 shutdown_connections 2024-04-26T21:09:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.276+0000 7f05fe400640 1 -- 172.21.15.26:0/2259419200 wait complete. 2024-04-26T21:09:45.295 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:08:31.322511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:09:45.296 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:09:45.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.514+0000 7fac9be00640 1 Processor -- start 2024-04-26T21:09:45.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.514+0000 7fac9be00640 1 -- start start 2024-04-26T21:09:45.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.516+0000 7fac9be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac9c073530 0x7fac9c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.516+0000 7fac9be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fac9c06e210 0x7fac9c0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.516+0000 7fac9be00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac9c06fbb0 con 0x7fac9c15bdc0 2024-04-26T21:09:45.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.516+0000 7fac9be00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fac9c0b4cf0 con 0x7fac9c06e210 2024-04-26T21:09:45.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.516+0000 7fac9be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fac9c0cf280 con 0x7fac9c073530 2024-04-26T21:09:45.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.516+0000 7fac9b400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fac9c06e210 0x7fac9c0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.517+0000 7fac9b400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fac9c06e210 0x7fac9c0cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48064/0 (socket says 172.21.15.26:48064) 2024-04-26T21:09:45.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.517+0000 7fac9b400640 1 -- 172.21.15.26:0/1770910450 learned_addr learned my addr 172.21.15.26:0/1770910450 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:45.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.517+0000 7fac9a000640 1 --2- 172.21.15.26:0/1770910450 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac9c073530 0x7fac9c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.517+0000 7fac9b400640 1 -- 172.21.15.26:0/1770910450 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac9c073530 msgr2=0x7fac9c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:45.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.517+0000 7fac9b400640 1 --2- 172.21.15.26:0/1770910450 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac9c073530 0x7fac9c0b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.517+0000 7fac9b400640 1 -- 172.21.15.26:0/1770910450 >> v1:172.21.15.86:6789/0 conn(0x7fac9c15bdc0 legacy=0x7fac9c072e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:45.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.517+0000 7fac9b400640 1 -- 172.21.15.26:0/1770910450 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c06e780 con 0x7fac9c06e210 2024-04-26T21:09:45.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.518+0000 7fac9b400640 1 --2- 172.21.15.26:0/1770910450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac9c06e210 0x7fac9c0cf710 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fac9400d150 tx=0x7fac9402c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e469ce3fab23d949 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:45.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.519+0000 7fac99600640 1 -- 172.21.15.26:0/1770910450 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94039070 con 0x7fac9c06e210 2024-04-26T21:09:45.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.519+0000 7fac99600640 1 -- 172.21.15.26:0/1770910450 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac94009600 con 0x7fac9c06e210 2024-04-26T21:09:45.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.519+0000 7fac99600640 1 -- 172.21.15.26:0/1770910450 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac940099d0 con 0x7fac9c06e210 2024-04-26T21:09:45.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.519+0000 7fac9be00640 1 -- 172.21.15.26:0/1770910450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac9c06e210 msgr2=0x7fac9c0cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:45.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.519+0000 7fac9be00640 1 --2- 172.21.15.26:0/1770910450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac9c06e210 0x7fac9c0cf710 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fac9400d150 tx=0x7fac9402c890 comp rx=0 tx=0).stop 2024-04-26T21:09:45.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.519+0000 7fac9be00640 1 -- 172.21.15.26:0/1770910450 shutdown_connections 2024-04-26T21:09:45.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.519+0000 7fac9be00640 1 --2- 172.21.15.26:0/1770910450 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac9c073530 0x7fac9c0b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.519+0000 7fac9be00640 1 --2- 172.21.15.26:0/1770910450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac9c06e210 0x7fac9c0cf710 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.519+0000 7fac9be00640 1 -- 172.21.15.26:0/1770910450 >> 172.21.15.26:0/1770910450 conn(0x7fac9c0b2a10 msgr2=0x7fac9c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.520+0000 7fac9be00640 1 -- 172.21.15.26:0/1770910450 shutdown_connections 2024-04-26T21:09:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.520+0000 7fac9be00640 1 -- 172.21.15.26:0/1770910450 wait complete. 2024-04-26T21:09:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.521+0000 7fac9be00640 1 Processor -- start 2024-04-26T21:09:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.521+0000 7fac9be00640 1 -- start start 2024-04-26T21:09:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.522+0000 7fac9be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac9c06e210 0x7fac9c0b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.522+0000 7fac9be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac9c073530 0x7fac9c141df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.522+0000 7fac9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac9c15bdc0 0x7fac9c142330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.522+0000 7fac9be00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c072380 con 0x7fac9c073530 2024-04-26T21:09:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.522+0000 7fac9be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c070130 con 0x7fac9c15bdc0 2024-04-26T21:09:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.522+0000 7fac9be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fac9c15c440 con 0x7fac9c06e210 2024-04-26T21:09:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.523+0000 7fac9a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac9c073530 0x7fac9c141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.523+0000 7fac9a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac9c073530 0x7fac9c141df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48066/0 (socket says 172.21.15.26:48066) 2024-04-26T21:09:45.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.523+0000 7fac9a000640 1 -- 172.21.15.26:0/1186570014 learned_addr learned my addr 172.21.15.26:0/1186570014 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:45.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.523+0000 7fac9b400640 1 --2- 172.21.15.26:0/1186570014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac9c15bdc0 0x7fac9c142330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.523+0000 7fac9aa00640 1 --2- 172.21.15.26:0/1186570014 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac9c06e210 0x7fac9c0b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.523+0000 7fac9a000640 1 -- 172.21.15.26:0/1186570014 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac9c06e210 msgr2=0x7fac9c0b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:45.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.523+0000 7fac9a000640 1 --2- 172.21.15.26:0/1186570014 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac9c06e210 0x7fac9c0b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.523+0000 7fac9a000640 1 -- 172.21.15.26:0/1186570014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac9c15bdc0 msgr2=0x7fac9c142330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:45.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.523+0000 7fac9a000640 1 --2- 172.21.15.26:0/1186570014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac9c15bdc0 0x7fac9c142330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.523+0000 7fac9a000640 1 -- 172.21.15.26:0/1186570014 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c15c720 con 0x7fac9c073530 2024-04-26T21:09:45.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.523+0000 7fac9aa00640 1 --2- 172.21.15.26:0/1186570014 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac9c06e210 0x7fac9c0b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:45.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.524+0000 7fac9a000640 1 --2- 172.21.15.26:0/1186570014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac9c073530 0x7fac9c141df0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fac8800e9b0 tx=0x7fac8800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:45.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.524+0000 7fac92a00640 1 -- 172.21.15.26:0/1186570014 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac88017070 con 0x7fac9c073530 2024-04-26T21:09:45.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.524+0000 7fac92a00640 1 -- 172.21.15.26:0/1186570014 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac8800c9a0 con 0x7fac9c073530 2024-04-26T21:09:45.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.524+0000 7fac92a00640 1 -- 172.21.15.26:0/1186570014 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8800cd90 con 0x7fac9c073530 2024-04-26T21:09:45.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.524+0000 7fac9be00640 1 -- 172.21.15.26:0/1186570014 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9c0b11f0 con 0x7fac9c073530 2024-04-26T21:09:45.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.524+0000 7fac9be00640 1 -- 172.21.15.26:0/1186570014 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac9c142870 con 0x7fac9c073530 2024-04-26T21:09:45.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.526+0000 7fac9be00640 1 -- 172.21.15.26:0/1186570014 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac9c06a700 con 0x7fac9c073530 2024-04-26T21:09:45.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.526+0000 7fac92a00640 1 -- 172.21.15.26:0/1186570014 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac8801a070 con 0x7fac9c073530 2024-04-26T21:09:45.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.527+0000 7fac92a00640 1 --2- 172.21.15.26:0/1186570014 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fac7c06c940 0x7fac7c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.527+0000 7fac92a00640 1 -- 172.21.15.26:0/1186570014 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fac8808ba20 con 0x7fac9c073530 2024-04-26T21:09:45.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.528+0000 7fac9aa00640 1 --2- 172.21.15.26:0/1186570014 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fac7c06c940 0x7fac7c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.528+0000 7fac9aa00640 1 --2- 172.21.15.26:0/1186570014 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fac7c06c940 0x7fac7c06ee00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fac8c000f60 tx=0x7fac8c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:45.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.532+0000 7fac92a00640 1 -- 172.21.15.26:0/1186570014 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fac8805a310 con 0x7fac9c073530 2024-04-26T21:09:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.737+0000 7fac9be00640 1 -- 172.21.15.26:0/1186570014 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fac9c0d0610 con 0x7fac9c073530 2024-04-26T21:09:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.738+0000 7fac92a00640 1 -- 172.21.15.26:0/1186570014 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) ==== 93+0+10 (secure 0 0 0) 0x7fac88059c10 con 0x7fac9c073530 2024-04-26T21:09:45.738 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-26T21:09:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.740+0000 7fac9be00640 1 -- 172.21.15.26:0/1186570014 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fac7c06c940 msgr2=0x7fac7c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.740+0000 7fac9be00640 1 --2- 172.21.15.26:0/1186570014 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fac7c06c940 0x7fac7c06ee00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fac8c000f60 tx=0x7fac8c009040 comp rx=0 tx=0).stop 2024-04-26T21:09:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.740+0000 7fac9be00640 1 -- 172.21.15.26:0/1186570014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac9c073530 msgr2=0x7fac9c141df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.740+0000 7fac9be00640 1 --2- 172.21.15.26:0/1186570014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac9c073530 0x7fac9c141df0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fac8800e9b0 tx=0x7fac8800ee70 comp rx=0 tx=0).stop 2024-04-26T21:09:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.740+0000 7fac9be00640 1 -- 172.21.15.26:0/1186570014 shutdown_connections 2024-04-26T21:09:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.740+0000 7fac9be00640 1 --2- 172.21.15.26:0/1186570014 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fac7c06c940 0x7fac7c06ee00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.740+0000 7fac9be00640 1 --2- 172.21.15.26:0/1186570014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac9c15bdc0 0x7fac9c142330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.740+0000 7fac9be00640 1 --2- 172.21.15.26:0/1186570014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac9c073530 0x7fac9c141df0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.740+0000 7fac9be00640 1 --2- 172.21.15.26:0/1186570014 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac9c06e210 0x7fac9c0b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.740+0000 7fac9be00640 1 -- 172.21.15.26:0/1186570014 >> 172.21.15.26:0/1186570014 conn(0x7fac9c0b2a10 msgr2=0x7fac9c0d1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.740+0000 7fac9be00640 1 -- 172.21.15.26:0/1186570014 shutdown_connections 2024-04-26T21:09:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.740+0000 7fac9be00640 1 -- 172.21.15.26:0/1186570014 wait complete. 2024-04-26T21:09:45.758 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:09:45.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.957+0000 7fc8c2800640 1 Processor -- start 2024-04-26T21:09:45.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.957+0000 7fc8c2800640 1 -- start start 2024-04-26T21:09:45.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.959+0000 7fc8c2800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8bc146830 0x7fc8bc15d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.959+0000 7fc8c2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc8bc0b2e30 0x7fc8bc0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.959+0000 7fc8c2800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8bc0b4280 con 0x7fc8bc0b3d10 2024-04-26T21:09:45.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.959+0000 7fc8c2800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc8bc146c10 con 0x7fc8bc0b2e30 2024-04-26T21:09:45.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.959+0000 7fc8c2800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc8bc0b4560 con 0x7fc8bc146830 2024-04-26T21:09:45.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.959+0000 7fc8c0a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc8bc0b3d10 0x7fc8bc073f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32944/0 (socket says 172.21.15.26:32944) 2024-04-26T21:09:45.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.959+0000 7fc8c0a00640 1 -- 172.21.15.26:0/3774701606 learned_addr learned my addr 172.21.15.26:0/3774701606 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:45.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.960+0000 7fc8c1400640 1 --2- 172.21.15.26:0/3774701606 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8bc146830 0x7fc8bc15d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.960+0000 7fc8c1e00640 1 --2- 172.21.15.26:0/3774701606 >> v2:172.21.15.86:3300/0 conn(0x7fc8bc0b2e30 0x7fc8bc0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.960+0000 7fc8bbe00640 1 -- 172.21.15.26:0/3774701606 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1320549871 0 0) 0x7fc8bc0b4280 con 0x7fc8bc0b3d10 2024-04-26T21:09:45.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.960+0000 7fc8bbe00640 1 -- 172.21.15.26:0/3774701606 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8a4003520 con 0x7fc8bc0b3d10 2024-04-26T21:09:45.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.960+0000 7fc8bbe00640 1 -- 172.21.15.26:0/3774701606 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 31762352 0 0) 0x7fc8a4003520 con 0x7fc8bc0b3d10 2024-04-26T21:09:45.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.960+0000 7fc8bbe00640 1 -- 172.21.15.26:0/3774701606 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8bc146830 msgr2=0x7fc8bc15d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:45.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.960+0000 7fc8bbe00640 1 --2- 172.21.15.26:0/3774701606 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8bc146830 0x7fc8bc15d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.960+0000 7fc8bbe00640 1 -- 172.21.15.26:0/3774701606 >> v2:172.21.15.86:3300/0 conn(0x7fc8bc0b2e30 msgr2=0x7fc8bc0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:45.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.960+0000 7fc8bbe00640 1 --2- 172.21.15.26:0/3774701606 >> v2:172.21.15.86:3300/0 conn(0x7fc8bc0b2e30 0x7fc8bc0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.960+0000 7fc8bbe00640 1 -- 172.21.15.26:0/3774701606 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8bc0b4840 con 0x7fc8bc0b3d10 2024-04-26T21:09:45.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.960+0000 7fc8bbe00640 1 -- 172.21.15.26:0/3774701606 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7fc8a8001670 con 0x7fc8bc0b3d10 2024-04-26T21:09:45.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.960+0000 7fc8bbe00640 1 -- 172.21.15.26:0/3774701606 >> v1:172.21.15.26:6789/0 conn(0x7fc8bc0b3d10 legacy=0x7fc8bc073f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:45.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.960+0000 7fc8bbe00640 1 --2- 172.21.15.26:0/3774701606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8a40039b0 0x7fc8a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.960+0000 7fc8bbe00640 1 --2- 172.21.15.26:0/3774701606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4004800 0x7fc8a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.960+0000 7fc8bbe00640 1 --2- 172.21.15.26:0/3774701606 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8bc146830 0x7fc8a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.960+0000 7fc8bbe00640 1 -- 172.21.15.26:0/3774701606 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc146c10 con 0x7fc8a40039b0 2024-04-26T21:09:45.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.960+0000 7fc8bbe00640 1 -- 172.21.15.26:0/3774701606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc0b4560 con 0x7fc8a4004800 2024-04-26T21:09:45.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.960+0000 7fc8bbe00640 1 -- 172.21.15.26:0/3774701606 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc8a400b9c0 con 0x7fc8bc146830 2024-04-26T21:09:45.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.961+0000 7fc8c1400640 1 --2- 172.21.15.26:0/3774701606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8a40039b0 0x7fc8a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.961+0000 7fc8c0a00640 1 --2- 172.21.15.26:0/3774701606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4004800 0x7fc8a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.961+0000 7fc8c1e00640 1 --2- 172.21.15.26:0/3774701606 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8bc146830 0x7fc8a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.961+0000 7fc8c1400640 1 -- 172.21.15.26:0/3774701606 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8bc146830 msgr2=0x7fc8a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:45.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.961+0000 7fc8c1400640 1 --2- 172.21.15.26:0/3774701606 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8bc146830 0x7fc8a40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.961+0000 7fc8c1400640 1 -- 172.21.15.26:0/3774701606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4004800 msgr2=0x7fc8a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:45.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.961+0000 7fc8c1400640 1 --2- 172.21.15.26:0/3774701606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4004800 0x7fc8a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.961+0000 7fc8c1400640 1 -- 172.21.15.26:0/3774701606 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a4003520 con 0x7fc8a40039b0 2024-04-26T21:09:45.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.961+0000 7fc8c0a00640 1 --2- 172.21.15.26:0/3774701606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a4004800 0x7fc8a4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:45.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.962+0000 7fc8c1400640 1 --2- 172.21.15.26:0/3774701606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8a40039b0 0x7fc8a4003dd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009fc0 tx=0x7fc8a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fafca05dd695ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:45.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.962+0000 7fc8bbe00640 1 -- 172.21.15.26:0/3774701606 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0009c10 con 0x7fc8a40039b0 2024-04-26T21:09:45.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.962+0000 7fc8bbe00640 1 -- 172.21.15.26:0/3774701606 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b0036070 con 0x7fc8a40039b0 2024-04-26T21:09:45.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.962+0000 7fc8bbe00640 1 -- 172.21.15.26:0/3774701606 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0031880 con 0x7fc8a40039b0 2024-04-26T21:09:45.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.963+0000 7fc8c2800640 1 -- 172.21.15.26:0/3774701606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8a40039b0 msgr2=0x7fc8a4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:45.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.963+0000 7fc8c2800640 1 --2- 172.21.15.26:0/3774701606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8a40039b0 0x7fc8a4003dd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009fc0 tx=0x7fc8a4005830 comp rx=0 tx=0).stop 2024-04-26T21:09:45.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.963+0000 7fc8c1e00640 1 -- 172.21.15.26:0/3774701606 reap_dead start 2024-04-26T21:09:45.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.963+0000 7fc8c2800640 1 -- 172.21.15.26:0/3774701606 shutdown_connections 2024-04-26T21:09:45.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.963+0000 7fc8c2800640 1 -- 172.21.15.26:0/3774701606 >> 172.21.15.26:0/3774701606 conn(0x7fc8bc06c660 msgr2=0x7fc8bc06ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:45.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.963+0000 7fc8c2800640 1 -- 172.21.15.26:0/3774701606 shutdown_connections 2024-04-26T21:09:45.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.963+0000 7fc8c2800640 1 -- 172.21.15.26:0/3774701606 wait complete. 2024-04-26T21:09:45.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.965+0000 7fc8c2800640 1 Processor -- start 2024-04-26T21:09:45.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.965+0000 7fc8c2800640 1 -- start start 2024-04-26T21:09:45.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.966+0000 7fc8c2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a40039b0 0x7fc8bc0ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.966+0000 7fc8c2800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8bc0cc2d0 0x7fc8bc13d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.966+0000 7fc8c2800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8bc15f230 0x7fc8bc15f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.966+0000 7fc8c2800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc15d6b0 con 0x7fc8bc0cc2d0 2024-04-26T21:09:45.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.966+0000 7fc8c2800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc0b4e00 con 0x7fc8a40039b0 2024-04-26T21:09:45.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.966+0000 7fc8c2800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc8bc15d830 con 0x7fc8bc15f230 2024-04-26T21:09:45.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.966+0000 7fc8c0a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8bc0cc2d0 0x7fc8bc13d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.966+0000 7fc8c0a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8bc0cc2d0 0x7fc8bc13d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48084/0 (socket says 172.21.15.26:48084) 2024-04-26T21:09:45.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.966+0000 7fc8c0a00640 1 -- 172.21.15.26:0/2462942116 learned_addr learned my addr 172.21.15.26:0/2462942116 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:45.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.967+0000 7fc8c1400640 1 --2- 172.21.15.26:0/2462942116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a40039b0 0x7fc8bc0ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.967+0000 7fc8c1e00640 1 --2- 172.21.15.26:0/2462942116 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8bc15f230 0x7fc8bc15f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.967+0000 7fc8c0a00640 1 -- 172.21.15.26:0/2462942116 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8bc15f230 msgr2=0x7fc8bc15f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:45.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.967+0000 7fc8c0a00640 1 --2- 172.21.15.26:0/2462942116 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8bc15f230 0x7fc8bc15f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.967+0000 7fc8c0a00640 1 -- 172.21.15.26:0/2462942116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a40039b0 msgr2=0x7fc8bc0ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:45.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.967+0000 7fc8c0a00640 1 --2- 172.21.15.26:0/2462942116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a40039b0 0x7fc8bc0ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:45.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.967+0000 7fc8c0a00640 1 -- 172.21.15.26:0/2462942116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8bc06f990 con 0x7fc8bc0cc2d0 2024-04-26T21:09:45.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.967+0000 7fc8c0a00640 1 --2- 172.21.15.26:0/2462942116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8bc0cc2d0 0x7fc8bc13d910 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800d840 tx=0x7fc8a800dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:45.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.967+0000 7fc8b9600640 1 -- 172.21.15.26:0/2462942116 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8019070 con 0x7fc8bc0cc2d0 2024-04-26T21:09:45.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.967+0000 7fc8b9600640 1 -- 172.21.15.26:0/2462942116 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8a80074e0 con 0x7fc8bc0cc2d0 2024-04-26T21:09:45.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.967+0000 7fc8c2800640 1 -- 172.21.15.26:0/2462942116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8bc0cc730 con 0x7fc8bc0cc2d0 2024-04-26T21:09:45.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.967+0000 7fc8b9600640 1 -- 172.21.15.26:0/2462942116 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8004a20 con 0x7fc8bc0cc2d0 2024-04-26T21:09:45.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.968+0000 7fc8c2800640 1 -- 172.21.15.26:0/2462942116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8bc0caa00 con 0x7fc8bc0cc2d0 2024-04-26T21:09:45.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.969+0000 7fc8c2800640 1 -- 172.21.15.26:0/2462942116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8bc06a700 con 0x7fc8bc0cc2d0 2024-04-26T21:09:45.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.970+0000 7fc8b9600640 1 -- 172.21.15.26:0/2462942116 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8a801c070 con 0x7fc8bc0cc2d0 2024-04-26T21:09:45.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.971+0000 7fc8b9600640 1 --2- 172.21.15.26:0/2462942116 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc89c06c8f0 0x7fc89c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:45.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.971+0000 7fc8b9600640 1 -- 172.21.15.26:0/2462942116 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fc8a808b330 con 0x7fc8bc0cc2d0 2024-04-26T21:09:45.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.971+0000 7fc8c1400640 1 --2- 172.21.15.26:0/2462942116 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc89c06c8f0 0x7fc89c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:45.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.972+0000 7fc8c1400640 1 --2- 172.21.15.26:0/2462942116 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc89c06c8f0 0x7fc89c06edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0035000 tx=0x7fc8b002f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:45.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:45.976+0000 7fc8b9600640 1 -- 172.21.15.26:0/2462942116 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc8bc06f990 con 0x7fc8bc0cc2d0 2024-04-26T21:09:46.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.178+0000 7fc8c2800640 1 -- 172.21.15.26:0/2462942116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc8bc144a20 con 0x7fc8bc0cc2d0 2024-04-26T21:09:46.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.179+0000 7fc8b9600640 1 -- 172.21.15.26:0/2462942116 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) ==== 84+0+65 (secure 0 0 0) 0x7fc8a8059a20 con 0x7fc8bc0cc2d0 2024-04-26T21:09:46.180 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:46.180 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-04-26T21:09:46.180 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:09:46.180 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-04-26T21:09:46.180 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-04-26T21:09:46.180 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:09:46.180 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-04-26T21:09:46.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.182+0000 7fc8c2800640 1 -- 172.21.15.26:0/2462942116 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc89c06c8f0 msgr2=0x7fc89c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:46.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.182+0000 7fc8c2800640 1 --2- 172.21.15.26:0/2462942116 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc89c06c8f0 0x7fc89c06edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0035000 tx=0x7fc8b002f000 comp rx=0 tx=0).stop 2024-04-26T21:09:46.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.182+0000 7fc8c2800640 1 -- 172.21.15.26:0/2462942116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8bc0cc2d0 msgr2=0x7fc8bc13d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:46.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.182+0000 7fc8c2800640 1 --2- 172.21.15.26:0/2462942116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8bc0cc2d0 0x7fc8bc13d910 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800d840 tx=0x7fc8a800dd00 comp rx=0 tx=0).stop 2024-04-26T21:09:46.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.182+0000 7fc8c2800640 1 -- 172.21.15.26:0/2462942116 shutdown_connections 2024-04-26T21:09:46.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.182+0000 7fc8c2800640 1 --2- 172.21.15.26:0/2462942116 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc89c06c8f0 0x7fc89c06edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:46.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.182+0000 7fc8c2800640 1 --2- 172.21.15.26:0/2462942116 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8bc15f230 0x7fc8bc15f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:46.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.182+0000 7fc8c2800640 1 --2- 172.21.15.26:0/2462942116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8bc0cc2d0 0x7fc8bc13d910 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:46.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.182+0000 7fc8c2800640 1 --2- 172.21.15.26:0/2462942116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a40039b0 0x7fc8bc0ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:46.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.182+0000 7fc8c2800640 1 -- 172.21.15.26:0/2462942116 >> 172.21.15.26:0/2462942116 conn(0x7fc8bc06c660 msgr2=0x7fc8bc0b5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:46.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.182+0000 7fc8c2800640 1 -- 172.21.15.26:0/2462942116 shutdown_connections 2024-04-26T21:09:46.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.182+0000 7fc8c2800640 1 -- 172.21.15.26:0/2462942116 wait complete. 2024-04-26T21:09:46.200 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:09:46.200 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:09:46.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.414+0000 7fe89be00640 1 Processor -- start 2024-04-26T21:09:46.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.415+0000 7fe89be00640 1 -- start start 2024-04-26T21:09:46.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.416+0000 7fe89be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8940d2df0 0x7fe8940d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:46.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.416+0000 7fe89be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe89415f7a0 0x7fe89406dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:46.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.417+0000 7fe89be00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe89415d230 con 0x7fe894070ed0 2024-04-26T21:09:46.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.417+0000 7fe89be00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fe8940d3710 con 0x7fe89415f7a0 2024-04-26T21:09:46.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.417+0000 7fe89be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe894160170 con 0x7fe8940d2df0 2024-04-26T21:09:46.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.417+0000 7fe899e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe89415f7a0 0x7fe89406dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:46.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.417+0000 7fe899e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe89415f7a0 0x7fe89406dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48098/0 (socket says 172.21.15.26:48098) 2024-04-26T21:09:46.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.417+0000 7fe899e00640 1 -- 172.21.15.26:0/323911906 learned_addr learned my addr 172.21.15.26:0/323911906 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:46.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.417+0000 7fe899400640 1 --2- 172.21.15.26:0/323911906 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8940d2df0 0x7fe8940d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:46.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.417+0000 7fe899e00640 1 -- 172.21.15.26:0/323911906 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8940d2df0 msgr2=0x7fe8940d31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.417+0000 7fe899e00640 1 --2- 172.21.15.26:0/323911906 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8940d2df0 0x7fe8940d31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.417+0000 7fe899e00640 1 -- 172.21.15.26:0/323911906 >> v1:172.21.15.86:6789/0 conn(0x7fe894070ed0 legacy=0x7fe8941609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.417+0000 7fe899e00640 1 -- 172.21.15.26:0/323911906 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe894160450 con 0x7fe89415f7a0 2024-04-26T21:09:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.417+0000 7fe899400640 1 --2- 172.21.15.26:0/323911906 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8940d2df0 0x7fe8940d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.418+0000 7fe899e00640 1 --2- 172.21.15.26:0/323911906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89415f7a0 0x7fe89406dfa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009870 tx=0x7fe88c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6c67dbda7df634 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.419+0000 7fe893e00640 1 -- 172.21.15.26:0/323911906 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c037070 con 0x7fe89415f7a0 2024-04-26T21:09:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.419+0000 7fe893e00640 1 -- 172.21.15.26:0/323911906 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe88c032440 con 0x7fe89415f7a0 2024-04-26T21:09:46.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.419+0000 7fe893e00640 1 -- 172.21.15.26:0/323911906 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c032810 con 0x7fe89415f7a0 2024-04-26T21:09:46.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.419+0000 7fe89be00640 1 -- 172.21.15.26:0/323911906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89415f7a0 msgr2=0x7fe89406dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:46.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.419+0000 7fe89be00640 1 --2- 172.21.15.26:0/323911906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89415f7a0 0x7fe89406dfa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe88c009870 tx=0x7fe88c02a210 comp rx=0 tx=0).stop 2024-04-26T21:09:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.420+0000 7fe89be00640 1 -- 172.21.15.26:0/323911906 shutdown_connections 2024-04-26T21:09:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.420+0000 7fe89be00640 1 --2- 172.21.15.26:0/323911906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89415f7a0 0x7fe89406dfa0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.420+0000 7fe89be00640 1 --2- 172.21.15.26:0/323911906 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8940d2df0 0x7fe8940d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.420+0000 7fe89be00640 1 -- 172.21.15.26:0/323911906 >> 172.21.15.26:0/323911906 conn(0x7fe8940b8370 msgr2=0x7fe8940b8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.420+0000 7fe89be00640 1 -- 172.21.15.26:0/323911906 shutdown_connections 2024-04-26T21:09:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.420+0000 7fe89be00640 1 -- 172.21.15.26:0/323911906 wait complete. 2024-04-26T21:09:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.421+0000 7fe89be00640 1 Processor -- start 2024-04-26T21:09:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.422+0000 7fe89be00640 1 -- start start 2024-04-26T21:09:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.423+0000 7fe89be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe894070ed0 0x7fe8940d2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.423+0000 7fe89be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8940d2df0 0x7fe894151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.423+0000 7fe89be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe89415f7a0 0x7fe8940d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.423+0000 7fe89be00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe894160730 con 0x7fe8940d2df0 2024-04-26T21:09:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.423+0000 7fe89be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe89406da80 con 0x7fe89415f7a0 2024-04-26T21:09:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.423+0000 7fe89be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe89415fc30 con 0x7fe894070ed0 2024-04-26T21:09:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.423+0000 7fe898a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8940d2df0 0x7fe894151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.423+0000 7fe898a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8940d2df0 0x7fe894151a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48104/0 (socket says 172.21.15.26:48104) 2024-04-26T21:09:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.423+0000 7fe898a00640 1 -- 172.21.15.26:0/1207541616 learned_addr learned my addr 172.21.15.26:0/1207541616 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.423+0000 7fe899400640 1 --2- 172.21.15.26:0/1207541616 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe894070ed0 0x7fe8940d2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.423+0000 7fe899e00640 1 --2- 172.21.15.26:0/1207541616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe89415f7a0 0x7fe8940d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.423+0000 7fe898a00640 1 -- 172.21.15.26:0/1207541616 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe894070ed0 msgr2=0x7fe8940d2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.423+0000 7fe898a00640 1 --2- 172.21.15.26:0/1207541616 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe894070ed0 0x7fe8940d2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.423+0000 7fe898a00640 1 -- 172.21.15.26:0/1207541616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe89415f7a0 msgr2=0x7fe8940d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.423+0000 7fe898a00640 1 --2- 172.21.15.26:0/1207541616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe89415f7a0 0x7fe8940d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.423+0000 7fe898a00640 1 -- 172.21.15.26:0/1207541616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8940d0c60 con 0x7fe8940d2df0 2024-04-26T21:09:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.424+0000 7fe898a00640 1 --2- 172.21.15.26:0/1207541616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8940d2df0 0x7fe894151a20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe88000c430 tx=0x7fe88000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.424+0000 7fe891600640 1 -- 172.21.15.26:0/1207541616 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880011790 con 0x7fe8940d2df0 2024-04-26T21:09:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.424+0000 7fe891600640 1 -- 172.21.15.26:0/1207541616 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe880017070 con 0x7fe8940d2df0 2024-04-26T21:09:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.424+0000 7fe89be00640 1 -- 172.21.15.26:0/1207541616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe894155440 con 0x7fe8940d2df0 2024-04-26T21:09:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.424+0000 7fe891600640 1 -- 172.21.15.26:0/1207541616 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880011bb0 con 0x7fe8940d2df0 2024-04-26T21:09:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.424+0000 7fe89be00640 1 -- 172.21.15.26:0/1207541616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe894155d60 con 0x7fe8940d2df0 2024-04-26T21:09:46.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.425+0000 7fe89be00640 1 -- 172.21.15.26:0/1207541616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe894155870 con 0x7fe8940d2df0 2024-04-26T21:09:46.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.426+0000 7fe891600640 1 -- 172.21.15.26:0/1207541616 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe88001c460 con 0x7fe8940d2df0 2024-04-26T21:09:46.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.427+0000 7fe891600640 1 --2- 172.21.15.26:0/1207541616 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe87406c8f0 0x7fe87406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:46.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.427+0000 7fe891600640 1 -- 172.21.15.26:0/1207541616 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fe88008b6d0 con 0x7fe8940d2df0 2024-04-26T21:09:46.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.428+0000 7fe899400640 1 --2- 172.21.15.26:0/1207541616 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe87406c8f0 0x7fe87406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.428+0000 7fe899400640 1 --2- 172.21.15.26:0/1207541616 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe87406c8f0 0x7fe87406edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe8880075b0 tx=0x7fe888007540 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:46.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.432+0000 7fe891600640 1 -- 172.21.15.26:0/1207541616 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe880059e40 con 0x7fe8940d2df0 2024-04-26T21:09:46.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:46.629+0000 7fe89be00640 1 -- 172.21.15.26:0/1207541616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe89406a700 con 0x7fe8940d2df0 2024-04-26T21:09:47.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.127+0000 7fe891600640 1 -- 172.21.15.26:0/1207541616 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) ==== 138+0+0 (secure 0 0 0) 0x7fe880059740 con 0x7fe8940d2df0 2024-04-26T21:09:47.128 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-04-26T21:09:47.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.131+0000 7fe89be00640 1 -- 172.21.15.26:0/1207541616 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe87406c8f0 msgr2=0x7fe87406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:47.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.131+0000 7fe89be00640 1 --2- 172.21.15.26:0/1207541616 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe87406c8f0 0x7fe87406edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe8880075b0 tx=0x7fe888007540 comp rx=0 tx=0).stop 2024-04-26T21:09:47.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.131+0000 7fe89be00640 1 -- 172.21.15.26:0/1207541616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8940d2df0 msgr2=0x7fe894151a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:47.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.131+0000 7fe89be00640 1 --2- 172.21.15.26:0/1207541616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8940d2df0 0x7fe894151a20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe88000c430 tx=0x7fe88000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.131+0000 7fe89be00640 1 -- 172.21.15.26:0/1207541616 shutdown_connections 2024-04-26T21:09:47.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.131+0000 7fe89be00640 1 --2- 172.21.15.26:0/1207541616 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe87406c8f0 0x7fe87406edb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.131+0000 7fe89be00640 1 --2- 172.21.15.26:0/1207541616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe89415f7a0 0x7fe8940d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.131+0000 7fe89be00640 1 --2- 172.21.15.26:0/1207541616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8940d2df0 0x7fe894151a20 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.131+0000 7fe89be00640 1 --2- 172.21.15.26:0/1207541616 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe894070ed0 0x7fe8940d2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.131+0000 7fe89be00640 1 -- 172.21.15.26:0/1207541616 >> 172.21.15.26:0/1207541616 conn(0x7fe8940b8370 msgr2=0x7fe8940b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:47.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.132+0000 7fe89be00640 1 -- 172.21.15.26:0/1207541616 shutdown_connections 2024-04-26T21:09:47.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.132+0000 7fe89be00640 1 -- 172.21.15.26:0/1207541616 wait complete. 2024-04-26T21:09:47.158 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:09:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.368+0000 7ff612800640 1 Processor -- start 2024-04-26T21:09:47.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.368+0000 7ff612800640 1 -- start start 2024-04-26T21:09:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.370+0000 7ff612800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff60c0cef80 0x7ff60c06e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.370+0000 7ff612800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff60c1598f0 0x7ff60c15fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.370+0000 7ff612800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff60c163c90 con 0x7ff60c159510 2024-04-26T21:09:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.370+0000 7ff612800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ff60c06e750 con 0x7ff60c0cef80 2024-04-26T21:09:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.370+0000 7ff612800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff60c0ae980 con 0x7ff60c1598f0 2024-04-26T21:09:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.370+0000 7ff611400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff60c159510 0x7ff60c0ce870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32950/0 (socket says 172.21.15.26:32950) 2024-04-26T21:09:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.370+0000 7ff611400640 1 -- 172.21.15.26:0/3517113896 learned_addr learned my addr 172.21.15.26:0/3517113896 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:47.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.371+0000 7ff610a00640 1 --2- 172.21.15.26:0/3517113896 >> v2:172.21.15.26:3300/0 conn(0x7ff60c0cef80 0x7ff60c06e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:47.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.371+0000 7ff611e00640 1 --2- 172.21.15.26:0/3517113896 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff60c1598f0 0x7ff60c15fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:47.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.371+0000 7ff60be00640 1 -- 172.21.15.26:0/3517113896 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4052289938 0 0) 0x7ff60c163c90 con 0x7ff60c159510 2024-04-26T21:09:47.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.371+0000 7ff60be00640 1 -- 172.21.15.26:0/3517113896 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5f4003520 con 0x7ff60c159510 2024-04-26T21:09:47.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.371+0000 7ff610a00640 1 -- 172.21.15.26:0/3517113896 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff60c1598f0 msgr2=0x7ff60c15fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:47.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.371+0000 7ff610a00640 1 --2- 172.21.15.26:0/3517113896 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff60c1598f0 0x7ff60c15fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.371+0000 7ff610a00640 1 -- 172.21.15.26:0/3517113896 >> v1:172.21.15.26:6789/0 conn(0x7ff60c159510 legacy=0x7ff60c0ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:47.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.371+0000 7ff610a00640 1 -- 172.21.15.26:0/3517113896 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60c0aec20 con 0x7ff60c0cef80 2024-04-26T21:09:47.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.372+0000 7ff610a00640 1 --2- 172.21.15.26:0/3517113896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff60c0cef80 0x7ff60c06e210 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff60c06e8d0 tx=0x7ff5f802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1825466a0dd1c54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:47.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.373+0000 7ff60be00640 1 -- 172.21.15.26:0/3517113896 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8037070 con 0x7ff60c0cef80 2024-04-26T21:09:47.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.373+0000 7ff60be00640 1 -- 172.21.15.26:0/3517113896 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5f802adb0 con 0x7ff60c0cef80 2024-04-26T21:09:47.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.373+0000 7ff60be00640 1 -- 172.21.15.26:0/3517113896 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8032660 con 0x7ff60c0cef80 2024-04-26T21:09:47.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.373+0000 7ff612800640 1 -- 172.21.15.26:0/3517113896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff60c0cef80 msgr2=0x7ff60c06e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:47.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.373+0000 7ff612800640 1 --2- 172.21.15.26:0/3517113896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff60c0cef80 0x7ff60c06e210 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff60c06e8d0 tx=0x7ff5f802a060 comp rx=0 tx=0).stop 2024-04-26T21:09:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.373+0000 7ff612800640 1 -- 172.21.15.26:0/3517113896 shutdown_connections 2024-04-26T21:09:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.373+0000 7ff612800640 1 --2- 172.21.15.26:0/3517113896 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff60c1598f0 0x7ff60c15fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.373+0000 7ff612800640 1 --2- 172.21.15.26:0/3517113896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff60c0cef80 0x7ff60c06e210 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.373+0000 7ff612800640 1 -- 172.21.15.26:0/3517113896 >> 172.21.15.26:0/3517113896 conn(0x7ff60c0b2a10 msgr2=0x7ff60c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.374+0000 7ff612800640 1 -- 172.21.15.26:0/3517113896 shutdown_connections 2024-04-26T21:09:47.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.374+0000 7ff612800640 1 -- 172.21.15.26:0/3517113896 wait complete. 2024-04-26T21:09:47.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.375+0000 7ff612800640 1 Processor -- start 2024-04-26T21:09:47.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.376+0000 7ff612800640 1 -- start start 2024-04-26T21:09:47.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.377+0000 7ff612800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff60c0cef80 0x7ff60c157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:47.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.377+0000 7ff612800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff60c159510 0x7ff60c15dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:47.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.377+0000 7ff612800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff60c1598f0 0x7ff60c06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:47.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.377+0000 7ff612800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c142420 con 0x7ff60c0cef80 2024-04-26T21:09:47.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.377+0000 7ff612800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c0af1e0 con 0x7ff60c1598f0 2024-04-26T21:09:47.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.377+0000 7ff612800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff60c1601c0 con 0x7ff60c159510 2024-04-26T21:09:47.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.377+0000 7ff611400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff60c0cef80 0x7ff60c157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:47.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.377+0000 7ff611400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff60c0cef80 0x7ff60c157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48124/0 (socket says 172.21.15.26:48124) 2024-04-26T21:09:47.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.377+0000 7ff611400640 1 -- 172.21.15.26:0/584223021 learned_addr learned my addr 172.21.15.26:0/584223021 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:47.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.377+0000 7ff610a00640 1 --2- 172.21.15.26:0/584223021 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff60c159510 0x7ff60c15dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:47.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.377+0000 7ff611e00640 1 --2- 172.21.15.26:0/584223021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff60c1598f0 0x7ff60c06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:47.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.377+0000 7ff611400640 1 -- 172.21.15.26:0/584223021 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff60c159510 msgr2=0x7ff60c15dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:47.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.377+0000 7ff611400640 1 --2- 172.21.15.26:0/584223021 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff60c159510 0x7ff60c15dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.377+0000 7ff611400640 1 -- 172.21.15.26:0/584223021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff60c1598f0 msgr2=0x7ff60c06efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:47.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.377+0000 7ff611400640 1 --2- 172.21.15.26:0/584223021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff60c1598f0 0x7ff60c06efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:47.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.378+0000 7ff611400640 1 -- 172.21.15.26:0/584223021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60c15ef20 con 0x7ff60c0cef80 2024-04-26T21:09:47.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.378+0000 7ff611e00640 1 --2- 172.21.15.26:0/584223021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff60c1598f0 0x7ff60c06efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:47.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.378+0000 7ff611400640 1 --2- 172.21.15.26:0/584223021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff60c0cef80 0x7ff60c157f70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff60000c430 tx=0x7ff60000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:47.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.378+0000 7ff609600640 1 -- 172.21.15.26:0/584223021 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600017070 con 0x7ff60c0cef80 2024-04-26T21:09:47.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.378+0000 7ff609600640 1 -- 172.21.15.26:0/584223021 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff600005ce0 con 0x7ff60c0cef80 2024-04-26T21:09:47.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.378+0000 7ff609600640 1 -- 172.21.15.26:0/584223021 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6000072f0 con 0x7ff60c0cef80 2024-04-26T21:09:47.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.378+0000 7ff612800640 1 -- 172.21.15.26:0/584223021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff60c15e6e0 con 0x7ff60c0cef80 2024-04-26T21:09:47.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.378+0000 7ff612800640 1 -- 172.21.15.26:0/584223021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff60c159e00 con 0x7ff60c0cef80 2024-04-26T21:09:47.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.379+0000 7ff612800640 1 -- 172.21.15.26:0/584223021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60c159fd0 con 0x7ff60c0cef80 2024-04-26T21:09:47.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.381+0000 7ff609600640 1 -- 172.21.15.26:0/584223021 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff60001a070 con 0x7ff60c0cef80 2024-04-26T21:09:47.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.382+0000 7ff609600640 1 --2- 172.21.15.26:0/584223021 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff5ec06c920 0x7ff5ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:47.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.382+0000 7ff609600640 1 -- 172.21.15.26:0/584223021 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5227+0+0 (secure 0 0 0) 0x7ff60008ae80 con 0x7ff60c0cef80 2024-04-26T21:09:47.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.382+0000 7ff610a00640 1 --2- 172.21.15.26:0/584223021 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff5ec06c920 0x7ff5ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:47.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.383+0000 7ff610a00640 1 --2- 172.21.15.26:0/584223021 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff5ec06c920 0x7ff5ec06ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8034000 tx=0x7ff5f802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:47.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.387+0000 7ff609600640 1 -- 172.21.15.26:0/584223021 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff60c15e6e0 con 0x7ff60c0cef80 2024-04-26T21:09:47.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:47.596+0000 7ff612800640 1 -- 172.21.15.26:0/584223021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff60c0744b0 con 0x7ff60c0cef80 2024-04-26T21:09:48.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.133+0000 7ff609600640 1 -- 172.21.15.26:0/584223021 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) ==== 170+0+0 (secure 0 0 0) 0x7ff6000593f0 con 0x7ff60c0cef80 2024-04-26T21:09:48.135 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-04-26T21:09:48.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.137+0000 7ff612800640 1 -- 172.21.15.26:0/584223021 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff5ec06c920 msgr2=0x7ff5ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:48.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.137+0000 7ff612800640 1 --2- 172.21.15.26:0/584223021 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff5ec06c920 0x7ff5ec06ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8034000 tx=0x7ff5f802e000 comp rx=0 tx=0).stop 2024-04-26T21:09:48.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.138+0000 7ff612800640 1 -- 172.21.15.26:0/584223021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff60c0cef80 msgr2=0x7ff60c157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:48.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.138+0000 7ff612800640 1 --2- 172.21.15.26:0/584223021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff60c0cef80 0x7ff60c157f70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff60000c430 tx=0x7ff60000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.138+0000 7ff612800640 1 -- 172.21.15.26:0/584223021 shutdown_connections 2024-04-26T21:09:48.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.138+0000 7ff612800640 1 --2- 172.21.15.26:0/584223021 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff5ec06c920 0x7ff5ec06ede0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.138+0000 7ff612800640 1 --2- 172.21.15.26:0/584223021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff60c1598f0 0x7ff60c06efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.138+0000 7ff612800640 1 --2- 172.21.15.26:0/584223021 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff60c159510 0x7ff60c15dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.138+0000 7ff612800640 1 --2- 172.21.15.26:0/584223021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff60c0cef80 0x7ff60c157f70 secure :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff60000c430 tx=0x7ff60000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.138+0000 7ff612800640 1 -- 172.21.15.26:0/584223021 >> 172.21.15.26:0/584223021 conn(0x7ff60c0b2a10 msgr2=0x7ff60c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:48.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.139+0000 7ff612800640 1 -- 172.21.15.26:0/584223021 shutdown_connections 2024-04-26T21:09:48.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.140+0000 7ff612800640 1 -- 172.21.15.26:0/584223021 wait complete. 2024-04-26T21:09:48.178 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:09:48.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.360+0000 7f3277600640 1 Processor -- start 2024-04-26T21:09:48.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.360+0000 7f3277600640 1 -- start start 2024-04-26T21:09:48.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.362+0000 7f3277600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f32700b5d30 0x7f32700b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:48.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.362+0000 7f3277600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f32700b42c0 0x7f32700b6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:48.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.362+0000 7f3277600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32700b6650 0x7f327006e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:48.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.362+0000 7f3277600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f3270073700 con 0x7f32700b5d30 2024-04-26T21:09:48.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.362+0000 7f3277600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f327006e420 con 0x7f32700b42c0 2024-04-26T21:09:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.362+0000 7f3277600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3270146c00 con 0x7f32700b6650 2024-04-26T21:09:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.362+0000 7f3274c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f32700b5d30 0x7f32700b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.362+0000 7f3274c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f32700b5d30 0x7f32700b3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48132/0 (socket says 172.21.15.26:48132) 2024-04-26T21:09:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.362+0000 7f3274c00640 1 -- 172.21.15.26:0/2730617313 learned_addr learned my addr 172.21.15.26:0/2730617313 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.362+0000 7f3275600640 1 --2- 172.21.15.26:0/2730617313 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32700b6650 0x7f327006e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.363+0000 7f326fe00640 1 --2- 172.21.15.26:0/2730617313 >> v2:172.21.15.86:3300/0 conn(0x7f32700b42c0 0x7f32700b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.363+0000 7f3274c00640 1 -- 172.21.15.26:0/2730617313 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32700b6650 msgr2=0x7f327006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.363+0000 7f3274c00640 1 --2- 172.21.15.26:0/2730617313 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32700b6650 0x7f327006e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.363+0000 7f3274c00640 1 -- 172.21.15.26:0/2730617313 >> v2:172.21.15.86:3300/0 conn(0x7f32700b42c0 msgr2=0x7f32700b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:48.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.363+0000 7f3274c00640 1 --2- 172.21.15.26:0/2730617313 >> v2:172.21.15.86:3300/0 conn(0x7f32700b42c0 0x7f32700b6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.363+0000 7f3274c00640 1 -- 172.21.15.26:0/2730617313 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3270072e60 con 0x7f32700b5d30 2024-04-26T21:09:48.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.363+0000 7f3275600640 1 --2- 172.21.15.26:0/2730617313 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32700b6650 0x7f327006e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:48.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.364+0000 7f3274c00640 1 --2- 172.21.15.26:0/2730617313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32700b5d30 0x7f32700b3d80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3270074520 tx=0x7f326402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd556dc8121643f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.364+0000 7f326f400640 1 -- 172.21.15.26:0/2730617313 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3264037070 con 0x7f32700b5d30 2024-04-26T21:09:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.364+0000 7f326f400640 1 -- 172.21.15.26:0/2730617313 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3264032440 con 0x7f32700b5d30 2024-04-26T21:09:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.364+0000 7f326f400640 1 -- 172.21.15.26:0/2730617313 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3264032810 con 0x7f32700b5d30 2024-04-26T21:09:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.365+0000 7f3277600640 1 -- 172.21.15.26:0/2730617313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32700b5d30 msgr2=0x7f32700b3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.365+0000 7f3277600640 1 --2- 172.21.15.26:0/2730617313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32700b5d30 0x7f32700b3d80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3270074520 tx=0x7f326402a1e0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.365+0000 7f3277600640 1 -- 172.21.15.26:0/2730617313 shutdown_connections 2024-04-26T21:09:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.365+0000 7f3277600640 1 --2- 172.21.15.26:0/2730617313 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32700b6650 0x7f327006e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.365+0000 7f3277600640 1 --2- 172.21.15.26:0/2730617313 >> v2:172.21.15.86:3300/0 conn(0x7f32700b42c0 0x7f32700b6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.365+0000 7f3277600640 1 --2- 172.21.15.26:0/2730617313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32700b5d30 0x7f32700b3d80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.365+0000 7f3277600640 1 -- 172.21.15.26:0/2730617313 >> 172.21.15.26:0/2730617313 conn(0x7f32700b2a80 msgr2=0x7f32700b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:48.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.365+0000 7f3277600640 1 -- 172.21.15.26:0/2730617313 shutdown_connections 2024-04-26T21:09:48.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.365+0000 7f3277600640 1 -- 172.21.15.26:0/2730617313 wait complete. 2024-04-26T21:09:48.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.367+0000 7f3277600640 1 Processor -- start 2024-04-26T21:09:48.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.367+0000 7f3277600640 1 -- start start 2024-04-26T21:09:48.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.368+0000 7f3277600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32700b42c0 0x7f32701ddf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:48.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.368+0000 7f3277600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32700b5d30 0x7f32701f3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:48.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.368+0000 7f3277600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32700b6650 0x7f3270144310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:48.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.368+0000 7f3277600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3270072a80 con 0x7f32700b5d30 2024-04-26T21:09:48.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.368+0000 7f3277600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3270073420 con 0x7f32700b6650 2024-04-26T21:09:48.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.368+0000 7f3277600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3270072c00 con 0x7f32700b42c0 2024-04-26T21:09:48.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.368+0000 7f326fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32700b5d30 0x7f32701f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:48.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.368+0000 7f326fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32700b5d30 0x7f32701f3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48140/0 (socket says 172.21.15.26:48140) 2024-04-26T21:09:48.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.368+0000 7f326fe00640 1 -- 172.21.15.26:0/2821546396 learned_addr learned my addr 172.21.15.26:0/2821546396 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:48.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.369+0000 7f3274c00640 1 --2- 172.21.15.26:0/2821546396 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32700b42c0 0x7f32701ddf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:48.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.369+0000 7f3275600640 1 --2- 172.21.15.26:0/2821546396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32700b6650 0x7f3270144310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:48.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.369+0000 7f326fe00640 1 -- 172.21.15.26:0/2821546396 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32700b42c0 msgr2=0x7f32701ddf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:48.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.369+0000 7f326fe00640 1 --2- 172.21.15.26:0/2821546396 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32700b42c0 0x7f32701ddf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.369+0000 7f326fe00640 1 -- 172.21.15.26:0/2821546396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32700b6650 msgr2=0x7f3270144310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:48.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.369+0000 7f326fe00640 1 --2- 172.21.15.26:0/2821546396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32700b6650 0x7f3270144310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:48.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.369+0000 7f326fe00640 1 -- 172.21.15.26:0/2821546396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32701de470 con 0x7f32700b5d30 2024-04-26T21:09:48.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.369+0000 7f3274c00640 1 --2- 172.21.15.26:0/2821546396 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32700b42c0 0x7f32701ddf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:48.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.369+0000 7f326fe00640 1 --2- 172.21.15.26:0/2821546396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32700b5d30 0x7f32701f3800 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f325c00c7f0 tx=0x7f325c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:48.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.369+0000 7f326cc00640 1 -- 172.21.15.26:0/2821546396 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c015070 con 0x7f32700b5d30 2024-04-26T21:09:48.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.369+0000 7f326cc00640 1 -- 172.21.15.26:0/2821546396 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f325c004220 con 0x7f32700b5d30 2024-04-26T21:09:48.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.369+0000 7f326cc00640 1 -- 172.21.15.26:0/2821546396 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c007620 con 0x7f32700b5d30 2024-04-26T21:09:48.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.369+0000 7f3277600640 1 -- 172.21.15.26:0/2821546396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32701f0b80 con 0x7f32700b5d30 2024-04-26T21:09:48.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.370+0000 7f3277600640 1 -- 172.21.15.26:0/2821546396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32701f0e20 con 0x7f32700b5d30 2024-04-26T21:09:48.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.371+0000 7f3277600640 1 -- 172.21.15.26:0/2821546396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f327006a700 con 0x7f32700b5d30 2024-04-26T21:09:48.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.372+0000 7f326cc00640 1 -- 172.21.15.26:0/2821546396 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f325c0079d0 con 0x7f32700b5d30 2024-04-26T21:09:48.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.373+0000 7f326cc00640 1 --2- 172.21.15.26:0/2821546396 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f325006c8f0 0x7f325006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:48.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.373+0000 7f326cc00640 1 -- 172.21.15.26:0/2821546396 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5609+0+0 (secure 0 0 0) 0x7f325c08bb90 con 0x7f32700b5d30 2024-04-26T21:09:48.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.373+0000 7f3274c00640 1 --2- 172.21.15.26:0/2821546396 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f325006c8f0 0x7f325006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:48.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.374+0000 7f3274c00640 1 --2- 172.21.15.26:0/2821546396 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f325006c8f0 0x7f325006edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3264034000 tx=0x7f326402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:48.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.378+0000 7f326cc00640 1 -- 172.21.15.26:0/2821546396 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f325c059f80 con 0x7f32700b5d30 2024-04-26T21:09:48.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:48.620+0000 7f3277600640 1 -- 172.21.15.26:0/2821546396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f327006d250 con 0x7f32700b5d30 2024-04-26T21:09:49.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.186+0000 7f326cc00640 1 -- 172.21.15.26:0/2821546396 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f325c059880 con 0x7f32700b5d30 2024-04-26T21:09:49.187 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 4 and data pool 5 2024-04-26T21:09:49.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.191+0000 7f3277600640 1 -- 172.21.15.26:0/2821546396 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f325006c8f0 msgr2=0x7f325006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:49.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.191+0000 7f3277600640 1 --2- 172.21.15.26:0/2821546396 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f325006c8f0 0x7f325006edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3264034000 tx=0x7f326402e000 comp rx=0 tx=0).stop 2024-04-26T21:09:49.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.191+0000 7f3277600640 1 -- 172.21.15.26:0/2821546396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32700b5d30 msgr2=0x7f32701f3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:49.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.191+0000 7f3277600640 1 --2- 172.21.15.26:0/2821546396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32700b5d30 0x7f32701f3800 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f325c00c7f0 tx=0x7f325c00ccb0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.192+0000 7f3277600640 1 -- 172.21.15.26:0/2821546396 shutdown_connections 2024-04-26T21:09:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.192+0000 7f3277600640 1 --2- 172.21.15.26:0/2821546396 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f325006c8f0 0x7f325006edb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.192+0000 7f3277600640 1 --2- 172.21.15.26:0/2821546396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32700b6650 0x7f3270144310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.192+0000 7f3277600640 1 --2- 172.21.15.26:0/2821546396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32700b5d30 0x7f32701f3800 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.192+0000 7f3277600640 1 --2- 172.21.15.26:0/2821546396 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32700b42c0 0x7f32701ddf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.192+0000 7f3277600640 1 -- 172.21.15.26:0/2821546396 >> 172.21.15.26:0/2821546396 conn(0x7f32700b2a80 msgr2=0x7f32700cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:49.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.193+0000 7f3277600640 1 -- 172.21.15.26:0/2821546396 shutdown_connections 2024-04-26T21:09:49.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.193+0000 7f3277600640 1 -- 172.21.15.26:0/2821546396 wait complete. 2024-04-26T21:09:49.224 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:09:49.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.396+0000 7feb5b000640 1 Processor -- start 2024-04-26T21:09:49.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.396+0000 7feb5b000640 1 -- start start 2024-04-26T21:09:49.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.397+0000 7feb5b000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feb540d1c80 0x7feb540d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:49.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.397+0000 7feb5b000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb540b61c0 con 0x7feb54162ba0 2024-04-26T21:09:49.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.397+0000 7feb5b000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb540b64a0 con 0x7feb540d2060 2024-04-26T21:09:49.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.397+0000 7feb5b000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7feb540b5f70 con 0x7feb540d1c80 2024-04-26T21:09:49.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.397+0000 7feb5a600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7feb54162ba0 0x7feb540c8b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50152/0 (socket says 172.21.15.26:50152) 2024-04-26T21:09:49.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.398+0000 7feb5a600640 1 -- 172.21.15.26:0/3841737927 learned_addr learned my addr 172.21.15.26:0/3841737927 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:49.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.398+0000 7feb59c00640 1 --2- 172.21.15.26:0/3841737927 >> v2:172.21.15.86:3300/0 conn(0x7feb540d1c80 0x7feb540d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:49.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.398+0000 7feb53e00640 1 -- 172.21.15.26:0/3841737927 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2647624200 0 0) 0x7feb540b61c0 con 0x7feb54162ba0 2024-04-26T21:09:49.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.398+0000 7feb53e00640 1 -- 172.21.15.26:0/3841737927 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb3c003520 con 0x7feb54162ba0 2024-04-26T21:09:49.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.398+0000 7feb53e00640 1 -- 172.21.15.26:0/3841737927 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 757969477 0 0) 0x7feb3c003520 con 0x7feb54162ba0 2024-04-26T21:09:49.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.398+0000 7feb53e00640 1 -- 172.21.15.26:0/3841737927 >> v2:172.21.15.86:3300/0 conn(0x7feb540d1c80 msgr2=0x7feb540d3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:49.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.398+0000 7feb53e00640 1 --2- 172.21.15.26:0/3841737927 >> v2:172.21.15.86:3300/0 conn(0x7feb540d1c80 0x7feb540d3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.398+0000 7feb53e00640 1 -- 172.21.15.26:0/3841737927 >> v1:172.21.15.86:6789/0 conn(0x7feb540d2060 legacy=0x7feb54162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:49.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.398+0000 7feb53e00640 1 -- 172.21.15.26:0/3841737927 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb54074610 con 0x7feb54162ba0 2024-04-26T21:09:49.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.398+0000 7feb53e00640 1 -- 172.21.15.26:0/3841737927 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7feb4c0019f0 con 0x7feb54162ba0 2024-04-26T21:09:49.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.398+0000 7feb53e00640 1 -- 172.21.15.26:0/3841737927 >> v1:172.21.15.26:6789/0 conn(0x7feb54162ba0 legacy=0x7feb540c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:49.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.399+0000 7feb53e00640 1 --2- 172.21.15.26:0/3841737927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb3c003a00 0x7feb3c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:49.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.399+0000 7feb53e00640 1 --2- 172.21.15.26:0/3841737927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb3c004800 0x7feb3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:49.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.399+0000 7feb53e00640 1 --2- 172.21.15.26:0/3841737927 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb3c005210 0x7feb3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:49.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.399+0000 7feb53e00640 1 -- 172.21.15.26:0/3841737927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7feb540b5f70 con 0x7feb3c003a00 2024-04-26T21:09:49.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.399+0000 7feb53e00640 1 -- 172.21.15.26:0/3841737927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c00c160 con 0x7feb3c004800 2024-04-26T21:09:49.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.399+0000 7feb53e00640 1 -- 172.21.15.26:0/3841737927 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7feb3c00c360 con 0x7feb3c005210 2024-04-26T21:09:49.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.399+0000 7feb59c00640 1 --2- 172.21.15.26:0/3841737927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb3c003a00 0x7feb3c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.399+0000 7feb59c00640 1 -- 172.21.15.26:0/3841737927 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb3c005210 msgr2=0x7feb3c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.399+0000 7feb59c00640 1 --2- 172.21.15.26:0/3841737927 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb3c005210 0x7feb3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.399+0000 7feb59c00640 1 -- 172.21.15.26:0/3841737927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb3c004800 msgr2=0x7feb3c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.399+0000 7feb59c00640 1 --2- 172.21.15.26:0/3841737927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb3c004800 0x7feb3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.399+0000 7feb59c00640 1 -- 172.21.15.26:0/3841737927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3c003520 con 0x7feb3c003a00 2024-04-26T21:09:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.399+0000 7feb5a600640 1 -- 172.21.15.26:0/3841737927 reap_dead start 2024-04-26T21:09:49.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.400+0000 7feb59c00640 1 --2- 172.21.15.26:0/3841737927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb3c003a00 0x7feb3c003e20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7feb48009a90 tx=0x7feb3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7278af36db2eb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:49.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.400+0000 7feb53e00640 1 -- 172.21.15.26:0/3841737927 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb48036070 con 0x7feb3c003a00 2024-04-26T21:09:49.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.400+0000 7feb53e00640 1 -- 172.21.15.26:0/3841737927 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb48031400 con 0x7feb3c003a00 2024-04-26T21:09:49.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.400+0000 7feb53e00640 1 -- 172.21.15.26:0/3841737927 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb48009690 con 0x7feb3c003a00 2024-04-26T21:09:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.401+0000 7feb5b000640 1 -- 172.21.15.26:0/3841737927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb3c003a00 msgr2=0x7feb3c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.401+0000 7feb5b000640 1 --2- 172.21.15.26:0/3841737927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb3c003a00 0x7feb3c003e20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7feb48009a90 tx=0x7feb3c005830 comp rx=0 tx=0).stop 2024-04-26T21:09:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.401+0000 7feb5b000640 1 -- 172.21.15.26:0/3841737927 shutdown_connections 2024-04-26T21:09:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.401+0000 7feb5b000640 1 --2- 172.21.15.26:0/3841737927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb3c003a00 0x7feb3c003e20 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.401+0000 7feb5b000640 1 -- 172.21.15.26:0/3841737927 >> 172.21.15.26:0/3841737927 conn(0x7feb540d5020 msgr2=0x7feb540d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.401+0000 7feb5b000640 1 -- 172.21.15.26:0/3841737927 shutdown_connections 2024-04-26T21:09:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.401+0000 7feb5b000640 1 -- 172.21.15.26:0/3841737927 wait complete. 2024-04-26T21:09:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.403+0000 7feb5b000640 1 Processor -- start 2024-04-26T21:09:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.403+0000 7feb5b000640 1 -- start start 2024-04-26T21:09:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.404+0000 7feb5b000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb3c003a00 0x7feb54151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.404+0000 7feb5b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb540b9650 0x7feb540b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.404+0000 7feb5b000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb540b9ab0 0x7feb540ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.404+0000 7feb5b000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7feb540d3830 con 0x7feb3c003a00 2024-04-26T21:09:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.404+0000 7feb5b000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb540d5d50 con 0x7feb540b9650 2024-04-26T21:09:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.404+0000 7feb5b000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7feb540d3cc0 con 0x7feb540b9ab0 2024-04-26T21:09:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.404+0000 7feb59c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb3c003a00 0x7feb54151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.404+0000 7feb59c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb3c003a00 0x7feb54151410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50984/0 (socket says 172.21.15.26:50984) 2024-04-26T21:09:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.404+0000 7feb59c00640 1 -- 172.21.15.26:0/1687136622 learned_addr learned my addr 172.21.15.26:0/1687136622 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.405+0000 7feb59200640 1 --2- 172.21.15.26:0/1687136622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb540b9650 0x7feb540b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.405+0000 7feb59c00640 1 -- 172.21.15.26:0/1687136622 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb540b9ab0 msgr2=0x7feb540ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.405+0000 7feb5a600640 1 --2- 172.21.15.26:0/1687136622 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb540b9ab0 0x7feb540ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.405+0000 7feb59c00640 1 --2- 172.21.15.26:0/1687136622 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb540b9ab0 0x7feb540ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.405+0000 7feb59c00640 1 -- 172.21.15.26:0/1687136622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb540b9650 msgr2=0x7feb540b7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.405+0000 7feb59c00640 1 --2- 172.21.15.26:0/1687136622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb540b9650 0x7feb540b7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.405+0000 7feb59c00640 1 -- 172.21.15.26:0/1687136622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb540b85b0 con 0x7feb3c003a00 2024-04-26T21:09:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.405+0000 7feb59c00640 1 --2- 172.21.15.26:0/1687136622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb3c003a00 0x7feb54151410 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7feb48034040 tx=0x7feb48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.405+0000 7feb51600640 1 -- 172.21.15.26:0/1687136622 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4803f020 con 0x7feb3c003a00 2024-04-26T21:09:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.405+0000 7feb51600640 1 -- 172.21.15.26:0/1687136622 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb48037040 con 0x7feb3c003a00 2024-04-26T21:09:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.405+0000 7feb51600640 1 -- 172.21.15.26:0/1687136622 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb48036070 con 0x7feb3c003a00 2024-04-26T21:09:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.405+0000 7feb5b000640 1 -- 172.21.15.26:0/1687136622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb540b8890 con 0x7feb3c003a00 2024-04-26T21:09:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.406+0000 7feb5b000640 1 -- 172.21.15.26:0/1687136622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb540b8e50 con 0x7feb3c003a00 2024-04-26T21:09:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.407+0000 7feb5b000640 1 -- 172.21.15.26:0/1687136622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb5406a700 con 0x7feb3c003a00 2024-04-26T21:09:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.408+0000 7feb51600640 1 -- 172.21.15.26:0/1687136622 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb48042020 con 0x7feb3c003a00 2024-04-26T21:09:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.408+0000 7feb51600640 1 --2- 172.21.15.26:0/1687136622 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7feb3406c8f0 0x7feb3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.408+0000 7feb51600640 1 -- 172.21.15.26:0/1687136622 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7feb480abc10 con 0x7feb3c003a00 2024-04-26T21:09:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.409+0000 7feb59200640 1 --2- 172.21.15.26:0/1687136622 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7feb3406c8f0 0x7feb3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.409+0000 7feb59200640 1 --2- 172.21.15.26:0/1687136622 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7feb3406c8f0 0x7feb3406edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7feb54154f80 tx=0x7feb40009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:49.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.413+0000 7feb51600640 1 -- 172.21.15.26:0/1687136622 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feb48049030 con 0x7feb3c003a00 2024-04-26T21:09:49.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.620+0000 7feb5b000640 1 -- 172.21.15.26:0/1687136622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb54161c90 con 0x7feb3c003a00 2024-04-26T21:09:49.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.622+0000 7feb51600640 1 -- 172.21.15.26:0/1687136622 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16634 (secure 0 0 0) 0x7feb4807a060 con 0x7feb3c003a00 2024-04-26T21:09:49.624 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:49.624 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":33,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:09:48.622095+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:46.646256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:47.613883+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:09:49.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.625+0000 7feb5b000640 1 -- 172.21.15.26:0/1687136622 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7feb3406c8f0 msgr2=0x7feb3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:49.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.625+0000 7feb5b000640 1 --2- 172.21.15.26:0/1687136622 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7feb3406c8f0 0x7feb3406edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7feb54154f80 tx=0x7feb40009040 comp rx=0 tx=0).stop 2024-04-26T21:09:49.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.625+0000 7feb5b000640 1 -- 172.21.15.26:0/1687136622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb3c003a00 msgr2=0x7feb54151410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:49.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.625+0000 7feb5b000640 1 --2- 172.21.15.26:0/1687136622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb3c003a00 0x7feb54151410 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7feb48034040 tx=0x7feb48004730 comp rx=0 tx=0).stop 2024-04-26T21:09:49.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.625+0000 7feb5b000640 1 -- 172.21.15.26:0/1687136622 shutdown_connections 2024-04-26T21:09:49.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.625+0000 7feb5b000640 1 --2- 172.21.15.26:0/1687136622 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7feb3406c8f0 0x7feb3406edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.625+0000 7feb5b000640 1 --2- 172.21.15.26:0/1687136622 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb540b9ab0 0x7feb540ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.625+0000 7feb5b000640 1 --2- 172.21.15.26:0/1687136622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb540b9650 0x7feb540b7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.625+0000 7feb5b000640 1 --2- 172.21.15.26:0/1687136622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb3c003a00 0x7feb54151410 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.625+0000 7feb5b000640 1 -- 172.21.15.26:0/1687136622 >> 172.21.15.26:0/1687136622 conn(0x7feb540d5020 msgr2=0x7feb5406e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:49.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.625+0000 7feb5b000640 1 -- 172.21.15.26:0/1687136622 shutdown_connections 2024-04-26T21:09:49.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.625+0000 7feb5b000640 1 -- 172.21.15.26:0/1687136622 wait complete. 2024-04-26T21:09:49.646 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:09:49.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.856+0000 7f62c8a00640 1 Processor -- start 2024-04-26T21:09:49.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.856+0000 7f62c8a00640 1 -- start start 2024-04-26T21:09:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.858+0000 7f62c8a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f62c40d06f0 0x7f62c40d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.858+0000 7f62c8a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f62c40d1670 0x7f62c4071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.858+0000 7f62c8a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62c40cf690 con 0x7f62c4145c20 2024-04-26T21:09:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.858+0000 7f62c8a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f62c40ced10 con 0x7f62c40d06f0 2024-04-26T21:09:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.858+0000 7f62c8a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f62c40d0b80 con 0x7f62c40d1670 2024-04-26T21:09:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.858+0000 7f62c1e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f62c40d06f0 0x7f62c40d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.858+0000 7f62c2800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f62c4145c20 0x7f62c4146450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50160/0 (socket says 172.21.15.26:50160) 2024-04-26T21:09:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.858+0000 7f62c2800640 1 -- 172.21.15.26:0/3298799516 learned_addr learned my addr 172.21.15.26:0/3298799516 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.859+0000 7f62c3200640 1 --2- 172.21.15.26:0/3298799516 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f62c40d1670 0x7f62c4071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.859+0000 7f62c1400640 1 -- 172.21.15.26:0/3298799516 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 791859868 0 0) 0x7f62c40cf690 con 0x7f62c4145c20 2024-04-26T21:09:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.859+0000 7f62c1400640 1 -- 172.21.15.26:0/3298799516 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62a8003520 con 0x7f62c4145c20 2024-04-26T21:09:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.859+0000 7f62c1e00640 1 -- 172.21.15.26:0/3298799516 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f62c40d1670 msgr2=0x7f62c4071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.859+0000 7f62c1e00640 1 --2- 172.21.15.26:0/3298799516 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f62c40d1670 0x7f62c4071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.859+0000 7f62c1e00640 1 -- 172.21.15.26:0/3298799516 >> v1:172.21.15.26:6789/0 conn(0x7f62c4145c20 legacy=0x7f62c4146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.859+0000 7f62c1e00640 1 -- 172.21.15.26:0/3298799516 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c40cf930 con 0x7f62c40d06f0 2024-04-26T21:09:49.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.860+0000 7f62c1e00640 1 --2- 172.21.15.26:0/3298799516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62c40d06f0 0x7f62c40d1130 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f62b00097a0 tx=0x7f62b002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f820c3de813efc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:49.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.860+0000 7f62c1400640 1 -- 172.21.15.26:0/3298799516 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b0037070 con 0x7f62c40d06f0 2024-04-26T21:09:49.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.860+0000 7f62c1400640 1 -- 172.21.15.26:0/3298799516 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b0032440 con 0x7f62c40d06f0 2024-04-26T21:09:49.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.860+0000 7f62c1400640 1 -- 172.21.15.26:0/3298799516 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b0032810 con 0x7f62c40d06f0 2024-04-26T21:09:49.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.861+0000 7f62c8a00640 1 -- 172.21.15.26:0/3298799516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62c40d06f0 msgr2=0x7f62c40d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:49.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.861+0000 7f62c8a00640 1 --2- 172.21.15.26:0/3298799516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62c40d06f0 0x7f62c40d1130 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f62b00097a0 tx=0x7f62b002a180 comp rx=0 tx=0).stop 2024-04-26T21:09:49.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.861+0000 7f62c8a00640 1 -- 172.21.15.26:0/3298799516 shutdown_connections 2024-04-26T21:09:49.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.861+0000 7f62c8a00640 1 --2- 172.21.15.26:0/3298799516 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f62c40d1670 0x7f62c4071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.861+0000 7f62c8a00640 1 --2- 172.21.15.26:0/3298799516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62c40d06f0 0x7f62c40d1130 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.861+0000 7f62c8a00640 1 -- 172.21.15.26:0/3298799516 >> 172.21.15.26:0/3298799516 conn(0x7f62c406c690 msgr2=0x7f62c406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:49.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.861+0000 7f62c8a00640 1 -- 172.21.15.26:0/3298799516 shutdown_connections 2024-04-26T21:09:49.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.861+0000 7f62c8a00640 1 -- 172.21.15.26:0/3298799516 wait complete. 2024-04-26T21:09:49.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.863+0000 7f62c8a00640 1 Processor -- start 2024-04-26T21:09:49.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.863+0000 7f62c8a00640 1 -- start start 2024-04-26T21:09:49.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.864+0000 7f62c8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c40d06f0 0x7f62c41f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:49.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.864+0000 7f62c8a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62c40d1670 0x7f62c41f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:49.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.864+0000 7f62c8a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f62c4145c20 0x7f62c40c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:49.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.864+0000 7f62c8a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f62c40cfef0 con 0x7f62c40d1670 2024-04-26T21:09:49.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.864+0000 7f62c8a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62c40d1aa0 con 0x7f62c40d06f0 2024-04-26T21:09:49.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.864+0000 7f62c8a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f62c40d01d0 con 0x7f62c4145c20 2024-04-26T21:09:49.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.864+0000 7f62c1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62c40d1670 0x7f62c41f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:49.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.864+0000 7f62c1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62c40d1670 0x7f62c41f5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51012/0 (socket says 172.21.15.26:51012) 2024-04-26T21:09:49.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.864+0000 7f62c1e00640 1 -- 172.21.15.26:0/841051491 learned_addr learned my addr 172.21.15.26:0/841051491 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:49.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.865+0000 7f62c3200640 1 --2- 172.21.15.26:0/841051491 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f62c4145c20 0x7f62c40c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:49.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.865+0000 7f62c2800640 1 --2- 172.21.15.26:0/841051491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c40d06f0 0x7f62c41f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:49.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.865+0000 7f62c1e00640 1 -- 172.21.15.26:0/841051491 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f62c4145c20 msgr2=0x7f62c40c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:49.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.865+0000 7f62c1e00640 1 --2- 172.21.15.26:0/841051491 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f62c4145c20 0x7f62c40c87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.865+0000 7f62c1e00640 1 -- 172.21.15.26:0/841051491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c40d06f0 msgr2=0x7f62c41f80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.865+0000 7f62c1e00640 1 --2- 172.21.15.26:0/841051491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c40d06f0 0x7f62c41f80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.865+0000 7f62c1e00640 1 -- 172.21.15.26:0/841051491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c41f61f0 con 0x7f62c40d1670 2024-04-26T21:09:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.865+0000 7f62c1e00640 1 --2- 172.21.15.26:0/841051491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62c40d1670 0x7f62c41f5cb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f62b0035040 tx=0x7f62b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.865+0000 7f62baa00640 1 -- 172.21.15.26:0/841051491 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b0037040 con 0x7f62c40d1670 2024-04-26T21:09:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.865+0000 7f62baa00640 1 -- 172.21.15.26:0/841051491 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b0030500 con 0x7f62c40d1670 2024-04-26T21:09:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.865+0000 7f62baa00640 1 -- 172.21.15.26:0/841051491 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b0030910 con 0x7f62c40d1670 2024-04-26T21:09:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.865+0000 7f62c8a00640 1 -- 172.21.15.26:0/841051491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c4143730 con 0x7f62c40d1670 2024-04-26T21:09:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.866+0000 7f62c8a00640 1 -- 172.21.15.26:0/841051491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c4143cf0 con 0x7f62c40d1670 2024-04-26T21:09:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.867+0000 7f62c8a00640 1 -- 172.21.15.26:0/841051491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62c40506c0 con 0x7f62c40d1670 2024-04-26T21:09:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.868+0000 7f62baa00640 1 -- 172.21.15.26:0/841051491 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f62b0044020 con 0x7f62c40d1670 2024-04-26T21:09:49.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.869+0000 7f62baa00640 1 --2- 172.21.15.26:0/841051491 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f62a406c8f0 0x7f62a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:49.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.869+0000 7f62baa00640 1 -- 172.21.15.26:0/841051491 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f62b00abe30 con 0x7f62c40d1670 2024-04-26T21:09:49.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.869+0000 7f62c2800640 1 --2- 172.21.15.26:0/841051491 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f62a406c8f0 0x7f62a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:49.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.870+0000 7f62c2800640 1 --2- 172.21.15.26:0/841051491 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f62a406c8f0 0x7f62a406edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f62c40d1290 tx=0x7f62b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:49.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:49.874+0000 7f62baa00640 1 -- 172.21.15.26:0/841051491 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f62c40506c0 con 0x7f62c40d1670 2024-04-26T21:09:50.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.077+0000 7f62c8a00640 1 -- 172.21.15.26:0/841051491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f62c4146000 con 0x7f62c40d1670 2024-04-26T21:09:50.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.080+0000 7f62baa00640 1 -- 172.21.15.26:0/841051491 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16634 (secure 0 0 0) 0x7f62c4146000 con 0x7f62c40d1670 2024-04-26T21:09:50.081 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:50.082 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":33,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:09:48.622095+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:46.646256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:47.613883+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:09:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.082+0000 7f62c8a00640 1 -- 172.21.15.26:0/841051491 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f62a406c8f0 msgr2=0x7f62a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.082+0000 7f62c8a00640 1 --2- 172.21.15.26:0/841051491 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f62a406c8f0 0x7f62a406edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f62c40d1290 tx=0x7f62b4009040 comp rx=0 tx=0).stop 2024-04-26T21:09:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.082+0000 7f62c8a00640 1 -- 172.21.15.26:0/841051491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62c40d1670 msgr2=0x7f62c41f5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.082+0000 7f62c8a00640 1 --2- 172.21.15.26:0/841051491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62c40d1670 0x7f62c41f5cb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f62b0035040 tx=0x7f62b00046c0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.082+0000 7f62c8a00640 1 -- 172.21.15.26:0/841051491 shutdown_connections 2024-04-26T21:09:50.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.082+0000 7f62c8a00640 1 --2- 172.21.15.26:0/841051491 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f62a406c8f0 0x7f62a406edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.082+0000 7f62c8a00640 1 --2- 172.21.15.26:0/841051491 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f62c4145c20 0x7f62c40c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.082+0000 7f62c8a00640 1 --2- 172.21.15.26:0/841051491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62c40d1670 0x7f62c41f5cb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.082+0000 7f62c8a00640 1 --2- 172.21.15.26:0/841051491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62c40d06f0 0x7f62c41f80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.082+0000 7f62c8a00640 1 -- 172.21.15.26:0/841051491 >> 172.21.15.26:0/841051491 conn(0x7f62c406c690 msgr2=0x7f62c40b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:50.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.083+0000 7f62c8a00640 1 -- 172.21.15.26:0/841051491 shutdown_connections 2024-04-26T21:09:50.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.083+0000 7f62c8a00640 1 -- 172.21.15.26:0/841051491 wait complete. 2024-04-26T21:09:50.104 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:09:50.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.317+0000 7ff1f5400640 1 Processor -- start 2024-04-26T21:09:50.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.318+0000 7ff1f5400640 1 -- start start 2024-04-26T21:09:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.319+0000 7ff1f5400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1f00b5d30 0x7ff1f00b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.319+0000 7ff1f5400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff1f006f8c0 0x7ff1f0166c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.319+0000 7ff1f5400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1f0146c00 con 0x7ff1f00b42c0 2024-04-26T21:09:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.319+0000 7ff1f5400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ff1f0073700 con 0x7ff1f006f8c0 2024-04-26T21:09:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.319+0000 7ff1f5400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f006e420 con 0x7ff1f00b5d30 2024-04-26T21:09:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.320+0000 7ff1ede00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff1f00b42c0 0x7ff1f00b6110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50176/0 (socket says 172.21.15.26:50176) 2024-04-26T21:09:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.320+0000 7ff1ede00640 1 -- 172.21.15.26:0/601571714 learned_addr learned my addr 172.21.15.26:0/601571714 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:50.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.320+0000 7ff1ef200640 1 --2- 172.21.15.26:0/601571714 >> v2:172.21.15.26:3300/0 conn(0x7ff1f006f8c0 0x7ff1f0166c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:50.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.320+0000 7ff1ee800640 1 --2- 172.21.15.26:0/601571714 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1f00b5d30 0x7ff1f00b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:50.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.320+0000 7ff1ed400640 1 -- 172.21.15.26:0/601571714 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2454719158 0 0) 0x7ff1f0146c00 con 0x7ff1f00b42c0 2024-04-26T21:09:50.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.320+0000 7ff1ed400640 1 -- 172.21.15.26:0/601571714 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1d4003520 con 0x7ff1f00b42c0 2024-04-26T21:09:50.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.320+0000 7ff1ef200640 1 -- 172.21.15.26:0/601571714 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1f00b5d30 msgr2=0x7ff1f00b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:50.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.320+0000 7ff1ef200640 1 --2- 172.21.15.26:0/601571714 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1f00b5d30 0x7ff1f00b3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.320+0000 7ff1ef200640 1 -- 172.21.15.26:0/601571714 >> v1:172.21.15.26:6789/0 conn(0x7ff1f00b42c0 legacy=0x7ff1f00b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:50.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.320+0000 7ff1ef200640 1 -- 172.21.15.26:0/601571714 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f0072e60 con 0x7ff1f006f8c0 2024-04-26T21:09:50.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.321+0000 7ff1ef200640 1 --2- 172.21.15.26:0/601571714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f006f8c0 0x7ff1f0166c90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80097a0 tx=0x7ff1e802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b2184ea7ebec91 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:50.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.322+0000 7ff1ed400640 1 -- 172.21.15.26:0/601571714 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8037070 con 0x7ff1f006f8c0 2024-04-26T21:09:50.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.322+0000 7ff1ed400640 1 -- 172.21.15.26:0/601571714 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e8032440 con 0x7ff1f006f8c0 2024-04-26T21:09:50.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.322+0000 7ff1ed400640 1 -- 172.21.15.26:0/601571714 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8032810 con 0x7ff1f006f8c0 2024-04-26T21:09:50.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.322+0000 7ff1f5400640 1 -- 172.21.15.26:0/601571714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f006f8c0 msgr2=0x7ff1f0166c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:50.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.322+0000 7ff1f5400640 1 --2- 172.21.15.26:0/601571714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f006f8c0 0x7ff1f0166c90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80097a0 tx=0x7ff1e802a180 comp rx=0 tx=0).stop 2024-04-26T21:09:50.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.322+0000 7ff1f5400640 1 -- 172.21.15.26:0/601571714 shutdown_connections 2024-04-26T21:09:50.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.322+0000 7ff1f5400640 1 --2- 172.21.15.26:0/601571714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f006f8c0 0x7ff1f0166c90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.322+0000 7ff1f5400640 1 --2- 172.21.15.26:0/601571714 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1f00b5d30 0x7ff1f00b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.322+0000 7ff1f5400640 1 -- 172.21.15.26:0/601571714 >> 172.21.15.26:0/601571714 conn(0x7ff1f00b2a80 msgr2=0x7ff1f00b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.323+0000 7ff1f5400640 1 -- 172.21.15.26:0/601571714 shutdown_connections 2024-04-26T21:09:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.323+0000 7ff1f5400640 1 -- 172.21.15.26:0/601571714 wait complete. 2024-04-26T21:09:50.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.324+0000 7ff1f5400640 1 Processor -- start 2024-04-26T21:09:50.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.325+0000 7ff1f5400640 1 -- start start 2024-04-26T21:09:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.326+0000 7ff1f5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1f006f8c0 0x7ff1f0142750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.326+0000 7ff1f5400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f00b42c0 0x7ff1f00cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.326+0000 7ff1f5400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1f00b5d30 0x7ff1f00cf810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.326+0000 7ff1f5400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f00b6820 con 0x7ff1f00b42c0 2024-04-26T21:09:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.326+0000 7ff1f5400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f0073420 con 0x7ff1f006f8c0 2024-04-26T21:09:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.326+0000 7ff1f5400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f0072a80 con 0x7ff1f00b5d30 2024-04-26T21:09:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.326+0000 7ff1ede00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f00b42c0 0x7ff1f00cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.326+0000 7ff1ede00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f00b42c0 0x7ff1f00cf2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51034/0 (socket says 172.21.15.26:51034) 2024-04-26T21:09:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.326+0000 7ff1ede00640 1 -- 172.21.15.26:0/481756092 learned_addr learned my addr 172.21.15.26:0/481756092 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.326+0000 7ff1ee800640 1 --2- 172.21.15.26:0/481756092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1f006f8c0 0x7ff1f0142750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.326+0000 7ff1ef200640 1 --2- 172.21.15.26:0/481756092 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1f00b5d30 0x7ff1f00cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.326+0000 7ff1ede00640 1 -- 172.21.15.26:0/481756092 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1f00b5d30 msgr2=0x7ff1f00cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.326+0000 7ff1ede00640 1 --2- 172.21.15.26:0/481756092 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1f00b5d30 0x7ff1f00cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.326+0000 7ff1ede00640 1 -- 172.21.15.26:0/481756092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1f006f8c0 msgr2=0x7ff1f0142750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.326+0000 7ff1ede00640 1 --2- 172.21.15.26:0/481756092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1f006f8c0 0x7ff1f0142750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.326+0000 7ff1ede00640 1 -- 172.21.15.26:0/481756092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f00cd220 con 0x7ff1f00b42c0 2024-04-26T21:09:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.326+0000 7ff1ee800640 1 --2- 172.21.15.26:0/481756092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1f006f8c0 0x7ff1f0142750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.327+0000 7ff1ef200640 1 --2- 172.21.15.26:0/481756092 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1f00b5d30 0x7ff1f00cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.327+0000 7ff1ede00640 1 --2- 172.21.15.26:0/481756092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f00b42c0 0x7ff1f00cf2d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800c430 tx=0x7ff1d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.327+0000 7ff1e6a00640 1 -- 172.21.15.26:0/481756092 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8017070 con 0x7ff1f00b42c0 2024-04-26T21:09:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.327+0000 7ff1e6a00640 1 -- 172.21.15.26:0/481756092 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1d8005ce0 con 0x7ff1f00b42c0 2024-04-26T21:09:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.327+0000 7ff1e6a00640 1 -- 172.21.15.26:0/481756092 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d80072f0 con 0x7ff1f00b42c0 2024-04-26T21:09:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.327+0000 7ff1f5400640 1 -- 172.21.15.26:0/481756092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f00cd500 con 0x7ff1f00b42c0 2024-04-26T21:09:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.327+0000 7ff1f5400640 1 -- 172.21.15.26:0/481756092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f00c7ac0 con 0x7ff1f00b42c0 2024-04-26T21:09:50.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.328+0000 7ff1f5400640 1 -- 172.21.15.26:0/481756092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f006a700 con 0x7ff1f00b42c0 2024-04-26T21:09:50.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.330+0000 7ff1e6a00640 1 -- 172.21.15.26:0/481756092 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff1d801a070 con 0x7ff1f00b42c0 2024-04-26T21:09:50.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.330+0000 7ff1e6a00640 1 --2- 172.21.15.26:0/481756092 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff1cc06c8f0 0x7ff1cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:50.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.331+0000 7ff1e6a00640 1 -- 172.21.15.26:0/481756092 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7ff1d808afe0 con 0x7ff1f00b42c0 2024-04-26T21:09:50.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.331+0000 7ff1ee800640 1 --2- 172.21.15.26:0/481756092 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff1cc06c8f0 0x7ff1cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:50.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.332+0000 7ff1ee800640 1 --2- 172.21.15.26:0/481756092 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff1cc06c8f0 0x7ff1cc06edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009f50 tx=0x7ff1e000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:50.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.335+0000 7ff1e6a00640 1 -- 172.21.15.26:0/481756092 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff1d8059460 con 0x7ff1f00b42c0 2024-04-26T21:09:50.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:50.577+0000 7ff1f5400640 1 -- 172.21.15.26:0/481756092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff1f00b61b0 con 0x7ff1f00b42c0 2024-04-26T21:09:51.088 INFO:tasks.ceph.mds.c.smithi026.stderr:2024-04-26T21:09:51.087+0000 7fdaff000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:09:51.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.202+0000 7ff1e6a00640 1 -- 172.21.15.26:0/481756092 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7ff1d8058d60 con 0x7ff1f00b42c0 2024-04-26T21:09:51.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.205+0000 7ff1f5400640 1 -- 172.21.15.26:0/481756092 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff1cc06c8f0 msgr2=0x7ff1cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.205+0000 7ff1f5400640 1 --2- 172.21.15.26:0/481756092 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff1cc06c8f0 0x7ff1cc06edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009f50 tx=0x7ff1e000b040 comp rx=0 tx=0).stop 2024-04-26T21:09:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.205+0000 7ff1f5400640 1 -- 172.21.15.26:0/481756092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f00b42c0 msgr2=0x7ff1f00cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.205+0000 7ff1f5400640 1 --2- 172.21.15.26:0/481756092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f00b42c0 0x7ff1f00cf2d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800c430 tx=0x7ff1d800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.206+0000 7ff1f5400640 1 -- 172.21.15.26:0/481756092 shutdown_connections 2024-04-26T21:09:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.206+0000 7ff1f5400640 1 --2- 172.21.15.26:0/481756092 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff1cc06c8f0 0x7ff1cc06edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.206+0000 7ff1f5400640 1 --2- 172.21.15.26:0/481756092 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1f00b5d30 0x7ff1f00cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.206+0000 7ff1f5400640 1 --2- 172.21.15.26:0/481756092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f00b42c0 0x7ff1f00cf2d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.206+0000 7ff1f5400640 1 --2- 172.21.15.26:0/481756092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1f006f8c0 0x7ff1f0142750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.206+0000 7ff1f5400640 1 -- 172.21.15.26:0/481756092 >> 172.21.15.26:0/481756092 conn(0x7ff1f00b2a80 msgr2=0x7ff1f00b22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.206+0000 7ff1f5400640 1 -- 172.21.15.26:0/481756092 shutdown_connections 2024-04-26T21:09:51.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.206+0000 7ff1f5400640 1 -- 172.21.15.26:0/481756092 wait complete. 2024-04-26T21:09:51.227 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T21:09:51.228 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T21:09:51.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.446+0000 7f50f3e00640 1 Processor -- start 2024-04-26T21:09:51.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.446+0000 7f50f3e00640 1 -- start start 2024-04-26T21:09:51.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.448+0000 7f50f3e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50f40b4040 0x7f50f406e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:51.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.448+0000 7f50f3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f50f4072490 0x7f50f40b6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:51.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.448+0000 7f50f3e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50f4072d90 con 0x7f50f4160a80 2024-04-26T21:09:51.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.448+0000 7f50f3e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f50f40b5be0 con 0x7f50f4072490 2024-04-26T21:09:51.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.448+0000 7f50f3e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f50f4073070 con 0x7f50f40b4040 2024-04-26T21:09:51.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.448+0000 7f50f2a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f50f4160a80 0x7f50f4160e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50178/0 (socket says 172.21.15.26:50178) 2024-04-26T21:09:51.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.449+0000 7f50f2a00640 1 -- 172.21.15.26:0/2504841662 learned_addr learned my addr 172.21.15.26:0/2504841662 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:51.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.449+0000 7f50ebe00640 1 --2- 172.21.15.26:0/2504841662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50f40b4040 0x7f50f406e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:51.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.449+0000 7f50f3400640 1 --2- 172.21.15.26:0/2504841662 >> v2:172.21.15.86:3300/0 conn(0x7f50f4072490 0x7f50f40b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:51.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.449+0000 7f50f2000640 1 -- 172.21.15.26:0/2504841662 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2065270133 0 0) 0x7f50f4072d90 con 0x7f50f4160a80 2024-04-26T21:09:51.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.449+0000 7f50f2000640 1 -- 172.21.15.26:0/2504841662 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50d8003520 con 0x7f50f4160a80 2024-04-26T21:09:51.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.450+0000 7f50f2000640 1 -- 172.21.15.26:0/2504841662 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1851932138 0 0) 0x7f50d8003520 con 0x7f50f4160a80 2024-04-26T21:09:51.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.450+0000 7f50f2000640 1 -- 172.21.15.26:0/2504841662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50f40b4040 msgr2=0x7f50f406e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:51.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.450+0000 7f50f2000640 1 --2- 172.21.15.26:0/2504841662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50f40b4040 0x7f50f406e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.450+0000 7f50f2000640 1 -- 172.21.15.26:0/2504841662 >> v2:172.21.15.86:3300/0 conn(0x7f50f4072490 msgr2=0x7f50f40b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:51.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.450+0000 7f50f2000640 1 --2- 172.21.15.26:0/2504841662 >> v2:172.21.15.86:3300/0 conn(0x7f50f4072490 0x7f50f40b6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.450+0000 7f50f2000640 1 -- 172.21.15.26:0/2504841662 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f4073350 con 0x7f50f4160a80 2024-04-26T21:09:51.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.450+0000 7f50f2000640 1 -- 172.21.15.26:0/2504841662 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f50e4001670 con 0x7f50f4160a80 2024-04-26T21:09:51.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.450+0000 7f50f2000640 1 -- 172.21.15.26:0/2504841662 >> v1:172.21.15.26:6789/0 conn(0x7f50f4160a80 legacy=0x7f50f4160e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:51.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.450+0000 7f50f2000640 1 --2- 172.21.15.26:0/2504841662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50f40b4040 0x7f50d80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:51.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.450+0000 7f50f2000640 1 --2- 172.21.15.26:0/2504841662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50d8003f80 0x7f50d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:51.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.450+0000 7f50f2000640 1 --2- 172.21.15.26:0/2504841662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50d8004d40 0x7f50d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:51.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.450+0000 7f50f2000640 1 -- 172.21.15.26:0/2504841662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f50f40b5be0 con 0x7f50d8003f80 2024-04-26T21:09:51.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.450+0000 7f50f2000640 1 -- 172.21.15.26:0/2504841662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4073070 con 0x7f50d8004d40 2024-04-26T21:09:51.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.450+0000 7f50f2000640 1 -- 172.21.15.26:0/2504841662 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f50d800b9c0 con 0x7f50f40b4040 2024-04-26T21:09:51.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.450+0000 7f50ebe00640 1 --2- 172.21.15.26:0/2504841662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50d8003f80 0x7f50d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:51.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.450+0000 7f50f2a00640 1 --2- 172.21.15.26:0/2504841662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50f40b4040 0x7f50d80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:51.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.450+0000 7f50ebe00640 1 -- 172.21.15.26:0/2504841662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50f40b4040 msgr2=0x7f50d80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:51.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.450+0000 7f50ebe00640 1 --2- 172.21.15.26:0/2504841662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50f40b4040 0x7f50d80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.451+0000 7f50ebe00640 1 -- 172.21.15.26:0/2504841662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50d8004d40 msgr2=0x7f50d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:51.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.451+0000 7f50ebe00640 1 --2- 172.21.15.26:0/2504841662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50d8004d40 0x7f50d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.451+0000 7f50ebe00640 1 -- 172.21.15.26:0/2504841662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d8003520 con 0x7f50d8003f80 2024-04-26T21:09:51.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.452+0000 7f50ebe00640 1 --2- 172.21.15.26:0/2504841662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50d8003f80 0x7f50d8004800 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f50dc009af0 tx=0x7f50d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615b39edd304f342 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:51.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.452+0000 7f50f2000640 1 -- 172.21.15.26:0/2504841662 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc036070 con 0x7f50d8003f80 2024-04-26T21:09:51.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.452+0000 7f50f2000640 1 -- 172.21.15.26:0/2504841662 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50dc031400 con 0x7f50d8003f80 2024-04-26T21:09:51.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.452+0000 7f50f2000640 1 -- 172.21.15.26:0/2504841662 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc009690 con 0x7f50d8003f80 2024-04-26T21:09:51.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.452+0000 7f50f3e00640 1 -- 172.21.15.26:0/2504841662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50d8003f80 msgr2=0x7f50d8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.452+0000 7f50f3e00640 1 --2- 172.21.15.26:0/2504841662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50d8003f80 0x7f50d8004800 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f50dc009af0 tx=0x7f50d8005830 comp rx=0 tx=0).stop 2024-04-26T21:09:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.452+0000 7f50f3400640 1 -- 172.21.15.26:0/2504841662 reap_dead start 2024-04-26T21:09:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.452+0000 7f50f3e00640 1 -- 172.21.15.26:0/2504841662 shutdown_connections 2024-04-26T21:09:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.452+0000 7f50f3e00640 1 -- 172.21.15.26:0/2504841662 >> 172.21.15.26:0/2504841662 conn(0x7f50f40b2a10 msgr2=0x7f50f40b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.453+0000 7f50f3e00640 1 -- 172.21.15.26:0/2504841662 shutdown_connections 2024-04-26T21:09:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.453+0000 7f50f3e00640 1 -- 172.21.15.26:0/2504841662 wait complete. 2024-04-26T21:09:51.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.454+0000 7f50f3e00640 1 Processor -- start 2024-04-26T21:09:51.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.454+0000 7f50f3e00640 1 -- start start 2024-04-26T21:09:51.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.455+0000 7f50f3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50d8003f80 0x7f50f415c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:51.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.455+0000 7f50f3e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50f415b730 0x7f50f4155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:51.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.455+0000 7f50f3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f4157b50 0x7f50f40b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:51.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.455+0000 7f50f3e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4072aa0 con 0x7f50f4157b50 2024-04-26T21:09:51.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.455+0000 7f50f3e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4072920 con 0x7f50d8003f80 2024-04-26T21:09:51.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.455+0000 7f50f3e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f50f406e210 con 0x7f50f415b730 2024-04-26T21:09:51.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.456+0000 7f50f3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f4157b50 0x7f50f40b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:51.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.456+0000 7f50f3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f4157b50 0x7f50f40b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51044/0 (socket says 172.21.15.26:51044) 2024-04-26T21:09:51.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.456+0000 7f50f3400640 1 -- 172.21.15.26:0/3909160730 learned_addr learned my addr 172.21.15.26:0/3909160730 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:51.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.456+0000 7f50ebe00640 1 --2- 172.21.15.26:0/3909160730 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50f415b730 0x7f50f4155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:51.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.456+0000 7f50f3400640 1 -- 172.21.15.26:0/3909160730 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50f415b730 msgr2=0x7f50f4155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:51.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.456+0000 7f50f2a00640 1 --2- 172.21.15.26:0/3909160730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50d8003f80 0x7f50f415c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:51.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.456+0000 7f50f3400640 1 --2- 172.21.15.26:0/3909160730 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50f415b730 0x7f50f4155580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.456+0000 7f50f3400640 1 -- 172.21.15.26:0/3909160730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50d8003f80 msgr2=0x7f50f415c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:51.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.456+0000 7f50f3400640 1 --2- 172.21.15.26:0/3909160730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50d8003f80 0x7f50f415c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:51.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.456+0000 7f50f3400640 1 -- 172.21.15.26:0/3909160730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f415cf50 con 0x7f50f4157b50 2024-04-26T21:09:51.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.456+0000 7f50f3400640 1 --2- 172.21.15.26:0/3909160730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f4157b50 0x7f50f40b1350 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f50f4074bd0 tx=0x7f50ec00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:51.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.457+0000 7f50eaa00640 1 -- 172.21.15.26:0/3909160730 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec00bce0 con 0x7f50f4157b50 2024-04-26T21:09:51.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.457+0000 7f50eaa00640 1 -- 172.21.15.26:0/3909160730 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ec01a070 con 0x7f50f4157b50 2024-04-26T21:09:51.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.457+0000 7f50eaa00640 1 -- 172.21.15.26:0/3909160730 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec014640 con 0x7f50f4157b50 2024-04-26T21:09:51.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.457+0000 7f50f3e00640 1 -- 172.21.15.26:0/3909160730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f415cc70 con 0x7f50f4157b50 2024-04-26T21:09:51.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.457+0000 7f50f3e00640 1 -- 172.21.15.26:0/3909160730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f40b0b50 con 0x7f50f4157b50 2024-04-26T21:09:51.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.458+0000 7f50f3e00640 1 -- 172.21.15.26:0/3909160730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f406a700 con 0x7f50f4157b50 2024-04-26T21:09:51.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.459+0000 7f50eaa00640 1 -- 172.21.15.26:0/3909160730 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f50ec005d10 con 0x7f50f4157b50 2024-04-26T21:09:51.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.460+0000 7f50eaa00640 1 --2- 172.21.15.26:0/3909160730 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f50d006c920 0x7f50d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:51.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.460+0000 7f50eaa00640 1 -- 172.21.15.26:0/3909160730 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f50ec08bd20 con 0x7f50f4157b50 2024-04-26T21:09:51.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.460+0000 7f50f2a00640 1 --2- 172.21.15.26:0/3909160730 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f50d006c920 0x7f50d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:51.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.461+0000 7f50f2a00640 1 --2- 172.21.15.26:0/3909160730 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f50d006c920 0x7f50d006ede0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f50e4001eb0 tx=0x7f50e4000fd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:51.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.465+0000 7f50eaa00640 1 -- 172.21.15.26:0/3909160730 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f50ec05a0b0 con 0x7f50f4157b50 2024-04-26T21:09:51.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:51.719+0000 7f50f3e00640 1 -- 172.21.15.26:0/3909160730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f50f40b61c0 con 0x7f50f4157b50 2024-04-26T21:09:52.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.206+0000 7f50eaa00640 1 -- 172.21.15.26:0/3909160730 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f50ec0599b0 con 0x7f50f4157b50 2024-04-26T21:09:52.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.209+0000 7f50f3e00640 1 -- 172.21.15.26:0/3909160730 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f50d006c920 msgr2=0x7f50d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:52.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.209+0000 7f50f3e00640 1 --2- 172.21.15.26:0/3909160730 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f50d006c920 0x7f50d006ede0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f50e4001eb0 tx=0x7f50e4000fd0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.209+0000 7f50f3e00640 1 -- 172.21.15.26:0/3909160730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f4157b50 msgr2=0x7f50f40b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:52.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.209+0000 7f50f3e00640 1 --2- 172.21.15.26:0/3909160730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f4157b50 0x7f50f40b1350 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f50f4074bd0 tx=0x7f50ec00b480 comp rx=0 tx=0).stop 2024-04-26T21:09:52.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.210+0000 7f50f3e00640 1 -- 172.21.15.26:0/3909160730 shutdown_connections 2024-04-26T21:09:52.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.210+0000 7f50f3e00640 1 --2- 172.21.15.26:0/3909160730 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f50d006c920 0x7f50d006ede0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.210+0000 7f50f3e00640 1 --2- 172.21.15.26:0/3909160730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f4157b50 0x7f50f40b1350 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.210+0000 7f50f3e00640 1 --2- 172.21.15.26:0/3909160730 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f50f415b730 0x7f50f4155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.210+0000 7f50f3e00640 1 --2- 172.21.15.26:0/3909160730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50d8003f80 0x7f50f415c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.210+0000 7f50f3e00640 1 -- 172.21.15.26:0/3909160730 >> 172.21.15.26:0/3909160730 conn(0x7f50f40b2a10 msgr2=0x7f50f40b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:52.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.210+0000 7f50f3e00640 1 -- 172.21.15.26:0/3909160730 shutdown_connections 2024-04-26T21:09:52.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.210+0000 7f50f3e00640 1 -- 172.21.15.26:0/3909160730 wait complete. 2024-04-26T21:09:52.235 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T21:09:52.236 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:52.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.448+0000 7f9d49e00640 1 Processor -- start 2024-04-26T21:09:52.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.449+0000 7f9d49e00640 1 -- start start 2024-04-26T21:09:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.450+0000 7f9d49e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d440d1c80 0x7f9d440d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.450+0000 7f9d49e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9d44162ba0 0x7f9d440d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.450+0000 7f9d49e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d440b61c0 con 0x7f9d440d2060 2024-04-26T21:09:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.450+0000 7f9d49e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9d440b5f70 con 0x7f9d44162ba0 2024-04-26T21:09:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.450+0000 7f9d49e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9d440d3830 con 0x7f9d440d1c80 2024-04-26T21:09:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.451+0000 7f9d43e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9d440d2060 0x7f9d44162490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50190/0 (socket says 172.21.15.26:50190) 2024-04-26T21:09:52.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.451+0000 7f9d43e00640 1 -- 172.21.15.26:0/1332219671 learned_addr learned my addr 172.21.15.26:0/1332219671 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:52.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.451+0000 7f9d49400640 1 --2- 172.21.15.26:0/1332219671 >> v2:172.21.15.86:3300/0 conn(0x7f9d44162ba0 0x7f9d440d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:52.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.451+0000 7f9d48a00640 1 --2- 172.21.15.26:0/1332219671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d440d1c80 0x7f9d440d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.451+0000 7f9d43400640 1 -- 172.21.15.26:0/1332219671 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2830226011 0 0) 0x7f9d440b61c0 con 0x7f9d440d2060 2024-04-26T21:09:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.451+0000 7f9d43400640 1 -- 172.21.15.26:0/1332219671 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d2c003520 con 0x7f9d440d2060 2024-04-26T21:09:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.452+0000 7f9d43400640 1 -- 172.21.15.26:0/1332219671 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 88027473 0 0) 0x7f9d2c003520 con 0x7f9d440d2060 2024-04-26T21:09:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.452+0000 7f9d43400640 1 -- 172.21.15.26:0/1332219671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d440d1c80 msgr2=0x7f9d440d3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.452+0000 7f9d43400640 1 --2- 172.21.15.26:0/1332219671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d440d1c80 0x7f9d440d3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.452+0000 7f9d43400640 1 -- 172.21.15.26:0/1332219671 >> v2:172.21.15.86:3300/0 conn(0x7f9d44162ba0 msgr2=0x7f9d440d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.452+0000 7f9d43400640 1 --2- 172.21.15.26:0/1332219671 >> v2:172.21.15.86:3300/0 conn(0x7f9d44162ba0 0x7f9d440d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.452+0000 7f9d43400640 1 -- 172.21.15.26:0/1332219671 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d440b64a0 con 0x7f9d440d2060 2024-04-26T21:09:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.452+0000 7f9d43400640 1 -- 172.21.15.26:0/1332219671 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f9d30001670 con 0x7f9d440d2060 2024-04-26T21:09:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.452+0000 7f9d43400640 1 -- 172.21.15.26:0/1332219671 >> v1:172.21.15.26:6789/0 conn(0x7f9d440d2060 legacy=0x7f9d44162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.452+0000 7f9d43400640 1 --2- 172.21.15.26:0/1332219671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d2c0039b0 0x7f9d2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.452+0000 7f9d43400640 1 --2- 172.21.15.26:0/1332219671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d440d1c80 0x7f9d2c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.452+0000 7f9d43400640 1 --2- 172.21.15.26:0/1332219671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d2c004d40 0x7f9d2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.452+0000 7f9d43400640 1 -- 172.21.15.26:0/1332219671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9d440b5f70 con 0x7f9d2c0039b0 2024-04-26T21:09:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.452+0000 7f9d43400640 1 -- 172.21.15.26:0/1332219671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d440d3830 con 0x7f9d2c004d40 2024-04-26T21:09:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.452+0000 7f9d43400640 1 -- 172.21.15.26:0/1332219671 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9d2c00b9f0 con 0x7f9d440d1c80 2024-04-26T21:09:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.452+0000 7f9d48a00640 1 --2- 172.21.15.26:0/1332219671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d2c0039b0 0x7f9d2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.452+0000 7f9d43e00640 1 --2- 172.21.15.26:0/1332219671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d440d1c80 0x7f9d2c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.453+0000 7f9d49400640 1 --2- 172.21.15.26:0/1332219671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d2c004d40 0x7f9d2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.453+0000 7f9d48a00640 1 -- 172.21.15.26:0/1332219671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d440d1c80 msgr2=0x7f9d2c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.453+0000 7f9d48a00640 1 --2- 172.21.15.26:0/1332219671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d440d1c80 0x7f9d2c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.453+0000 7f9d48a00640 1 -- 172.21.15.26:0/1332219671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d2c004d40 msgr2=0x7f9d2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.453+0000 7f9d48a00640 1 --2- 172.21.15.26:0/1332219671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d2c004d40 0x7f9d2c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.453+0000 7f9d48a00640 1 -- 172.21.15.26:0/1332219671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2c003520 con 0x7f9d2c0039b0 2024-04-26T21:09:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.454+0000 7f9d48a00640 1 --2- 172.21.15.26:0/1332219671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d2c0039b0 0x7f9d2c003dd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009ac0 tx=0x7f9d2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6675db566d505bd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.454+0000 7f9d43400640 1 -- 172.21.15.26:0/1332219671 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38036070 con 0x7f9d2c0039b0 2024-04-26T21:09:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.454+0000 7f9d43400640 1 -- 172.21.15.26:0/1332219671 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d38031400 con 0x7f9d2c0039b0 2024-04-26T21:09:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.454+0000 7f9d43400640 1 -- 172.21.15.26:0/1332219671 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38009690 con 0x7f9d2c0039b0 2024-04-26T21:09:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.455+0000 7f9d49e00640 1 -- 172.21.15.26:0/1332219671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d2c0039b0 msgr2=0x7f9d2c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.455+0000 7f9d49e00640 1 --2- 172.21.15.26:0/1332219671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d2c0039b0 0x7f9d2c003dd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009ac0 tx=0x7f9d2c005830 comp rx=0 tx=0).stop 2024-04-26T21:09:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.455+0000 7f9d49400640 1 -- 172.21.15.26:0/1332219671 reap_dead start 2024-04-26T21:09:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.455+0000 7f9d49e00640 1 -- 172.21.15.26:0/1332219671 shutdown_connections 2024-04-26T21:09:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.455+0000 7f9d49e00640 1 -- 172.21.15.26:0/1332219671 >> 172.21.15.26:0/1332219671 conn(0x7f9d440d5020 msgr2=0x7f9d440d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.455+0000 7f9d49e00640 1 -- 172.21.15.26:0/1332219671 shutdown_connections 2024-04-26T21:09:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.455+0000 7f9d49e00640 1 -- 172.21.15.26:0/1332219671 wait complete. 2024-04-26T21:09:52.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.457+0000 7f9d49e00640 1 Processor -- start 2024-04-26T21:09:52.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.457+0000 7f9d49e00640 1 -- start start 2024-04-26T21:09:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.458+0000 7f9d49e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d2c0039b0 0x7f9d44150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.458+0000 7f9d49e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d441f5a30 0x7f9d441f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.458+0000 7f9d49e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d440ce230 0x7f9d441f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.458+0000 7f9d49e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9d440d5cc0 con 0x7f9d441f5a30 2024-04-26T21:09:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.458+0000 7f9d49e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d440d5b40 con 0x7f9d440ce230 2024-04-26T21:09:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.458+0000 7f9d49e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f9d440d3cc0 con 0x7f9d2c0039b0 2024-04-26T21:09:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.458+0000 7f9d43e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d441f5a30 0x7f9d441f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.458+0000 7f9d43e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d441f5a30 0x7f9d441f5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51058/0 (socket says 172.21.15.26:51058) 2024-04-26T21:09:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.458+0000 7f9d43e00640 1 -- 172.21.15.26:0/3068448388 learned_addr learned my addr 172.21.15.26:0/3068448388 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.458+0000 7f9d43e00640 1 -- 172.21.15.26:0/3068448388 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d2c0039b0 msgr2=0x7f9d44150590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.458+0000 7f9d43e00640 1 --2- 172.21.15.26:0/3068448388 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d2c0039b0 0x7f9d44150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.458+0000 7f9d43e00640 1 -- 172.21.15.26:0/3068448388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d440ce230 msgr2=0x7f9d441f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.458+0000 7f9d49400640 1 --2- 172.21.15.26:0/3068448388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d440ce230 0x7f9d441f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.458+0000 7f9d43e00640 1 --2- 172.21.15.26:0/3068448388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d440ce230 0x7f9d441f4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.459+0000 7f9d43e00640 1 -- 172.21.15.26:0/3068448388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d440b7de0 con 0x7f9d441f5a30 2024-04-26T21:09:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.459+0000 7f9d43e00640 1 --2- 172.21.15.26:0/3068448388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d441f5a30 0x7f9d441f5ef0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000e6d0 tx=0x7f9d3000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.459+0000 7f9d40c00640 1 -- 172.21.15.26:0/3068448388 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3000c660 con 0x7f9d441f5a30 2024-04-26T21:09:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.459+0000 7f9d40c00640 1 -- 172.21.15.26:0/3068448388 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d30018070 con 0x7f9d441f5a30 2024-04-26T21:09:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.459+0000 7f9d40c00640 1 -- 172.21.15.26:0/3068448388 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3000ca40 con 0x7f9d441f5a30 2024-04-26T21:09:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.459+0000 7f9d49e00640 1 -- 172.21.15.26:0/3068448388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d440b80c0 con 0x7f9d441f5a30 2024-04-26T21:09:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.459+0000 7f9d49e00640 1 -- 172.21.15.26:0/3068448388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d44150ad0 con 0x7f9d441f5a30 2024-04-26T21:09:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.461+0000 7f9d49e00640 1 -- 172.21.15.26:0/3068448388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d4406a700 con 0x7f9d441f5a30 2024-04-26T21:09:52.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.462+0000 7f9d40c00640 1 -- 172.21.15.26:0/3068448388 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d3000cbe0 con 0x7f9d441f5a30 2024-04-26T21:09:52.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.463+0000 7f9d40c00640 1 --2- 172.21.15.26:0/3068448388 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f9d2406c920 0x7f9d2406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:52.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.463+0000 7f9d40c00640 1 -- 172.21.15.26:0/3068448388 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f9d3008b5f0 con 0x7f9d441f5a30 2024-04-26T21:09:52.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.463+0000 7f9d48a00640 1 --2- 172.21.15.26:0/3068448388 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f9d2406c920 0x7f9d2406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:52.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.464+0000 7f9d48a00640 1 --2- 172.21.15.26:0/3068448388 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f9d2406c920 0x7f9d2406ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9d38033000 tx=0x7f9d3802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:52.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.469+0000 7f9d40c00640 1 -- 172.21.15.26:0/3068448388 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9d30059a20 con 0x7f9d441f5a30 2024-04-26T21:09:52.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.721+0000 7f9d49e00640 1 -- 172.21.15.26:0/3068448388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d44161850 con 0x7f9d441f5a30 2024-04-26T21:09:52.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.722+0000 7f9d40c00640 1 -- 172.21.15.26:0/3068448388 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7f9d30059320 con 0x7f9d441f5a30 2024-04-26T21:09:52.723 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:52.723 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2349160700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2349160700},{"type":"v1","addr":"172.21.15.26:6837","nonce":2349160700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4598,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2824191762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2824191762},{"type":"v1","addr":"172.21.15.86:6834","nonce":2824191762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/263746310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":263746310},{"type":"v1","addr":"172.21.15.86:6835","nonce":263746310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:09:48.622025+0000","modified":"2024-04-26T21:09:51.721103+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5058":{"gid":5058,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6836/2528180854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2528180854},{"type":"v1","addr":"172.21.15.26:6836","nonce":2528180854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5058,"qdb_cluster":[5058]},"id":2}]} 2024-04-26T21:09:52.724 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 17 2024-04-26T21:09:52.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.725+0000 7f9d49e00640 1 -- 172.21.15.26:0/3068448388 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f9d2406c920 msgr2=0x7f9d2406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:52.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.725+0000 7f9d49e00640 1 --2- 172.21.15.26:0/3068448388 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f9d2406c920 0x7f9d2406ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9d38033000 tx=0x7f9d3802e000 comp rx=0 tx=0).stop 2024-04-26T21:09:52.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.725+0000 7f9d49e00640 1 -- 172.21.15.26:0/3068448388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d441f5a30 msgr2=0x7f9d441f5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:52.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.725+0000 7f9d49e00640 1 --2- 172.21.15.26:0/3068448388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d441f5a30 0x7f9d441f5ef0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000e6d0 tx=0x7f9d3000eb90 comp rx=0 tx=0).stop 2024-04-26T21:09:52.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.725+0000 7f9d49e00640 1 -- 172.21.15.26:0/3068448388 shutdown_connections 2024-04-26T21:09:52.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.725+0000 7f9d49e00640 1 --2- 172.21.15.26:0/3068448388 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f9d2406c920 0x7f9d2406ede0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.725+0000 7f9d49e00640 1 --2- 172.21.15.26:0/3068448388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d440ce230 0x7f9d441f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.725+0000 7f9d49e00640 1 --2- 172.21.15.26:0/3068448388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d441f5a30 0x7f9d441f5ef0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.725+0000 7f9d49e00640 1 --2- 172.21.15.26:0/3068448388 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9d2c0039b0 0x7f9d44150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.725+0000 7f9d49e00640 1 -- 172.21.15.26:0/3068448388 >> 172.21.15.26:0/3068448388 conn(0x7f9d440d5020 msgr2=0x7f9d4406e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:52.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.725+0000 7f9d49e00640 1 -- 172.21.15.26:0/3068448388 shutdown_connections 2024-04-26T21:09:52.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.726+0000 7f9d49e00640 1 -- 172.21.15.26:0/3068448388 wait complete. 2024-04-26T21:09:52.746 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:09:52.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.956+0000 7f8b91200640 1 Processor -- start 2024-04-26T21:09:52.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.957+0000 7f8b91200640 1 -- start start 2024-04-26T21:09:52.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.958+0000 7f8b91200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8b8c145c20 0x7f8b8c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:52.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.958+0000 7f8b91200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8b8c146450 0x7f8b8c0d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:52.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.958+0000 7f8b91200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8b8c1468b0 0x7f8b8c0d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:52.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.958+0000 7f8b91200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f8b8c0ced10 con 0x7f8b8c145c20 2024-04-26T21:09:52.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.958+0000 7f8b91200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8b8c0cf690 con 0x7f8b8c1468b0 2024-04-26T21:09:52.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.958+0000 7f8b91200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8b8c0cf930 con 0x7f8b8c146450 2024-04-26T21:09:52.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.958+0000 7f8b8b400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8b8c145c20 0x7f8b8c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:52.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.958+0000 7f8b8b400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8b8c145c20 0x7f8b8c0d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51064/0 (socket says 172.21.15.26:51064) 2024-04-26T21:09:52.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.958+0000 7f8b8b400640 1 -- 172.21.15.26:0/947922147 learned_addr learned my addr 172.21.15.26:0/947922147 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:52.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.959+0000 7f8b8be00640 1 --2- 172.21.15.26:0/947922147 >> v2:172.21.15.86:3300/0 conn(0x7f8b8c1468b0 0x7f8b8c0d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:52.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.959+0000 7f8b83e00640 1 --2- 172.21.15.26:0/947922147 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8b8c146450 0x7f8b8c0d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:52.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.959+0000 7f8b8b400640 1 -- 172.21.15.26:0/947922147 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8b8c146450 msgr2=0x7f8b8c0d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:52.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.959+0000 7f8b8b400640 1 --2- 172.21.15.26:0/947922147 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8b8c146450 0x7f8b8c0d1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.959+0000 7f8b8b400640 1 -- 172.21.15.26:0/947922147 >> v2:172.21.15.86:3300/0 conn(0x7f8b8c1468b0 msgr2=0x7f8b8c0d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:52.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.959+0000 7f8b8b400640 1 --2- 172.21.15.26:0/947922147 >> v2:172.21.15.86:3300/0 conn(0x7f8b8c1468b0 0x7f8b8c0d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.959+0000 7f8b8b400640 1 -- 172.21.15.26:0/947922147 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b8c0cfc10 con 0x7f8b8c145c20 2024-04-26T21:09:52.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.960+0000 7f8b8b400640 1 --2- 172.21.15.26:0/947922147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b8c145c20 0x7f8b8c0d06f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c146e40 tx=0x7f8b7c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb69123170c878a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:52.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.960+0000 7f8b8aa00640 1 -- 172.21.15.26:0/947922147 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c037070 con 0x7f8b8c145c20 2024-04-26T21:09:52.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.960+0000 7f8b8aa00640 1 -- 172.21.15.26:0/947922147 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b7c032440 con 0x7f8b8c145c20 2024-04-26T21:09:52.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.960+0000 7f8b8aa00640 1 -- 172.21.15.26:0/947922147 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c032810 con 0x7f8b8c145c20 2024-04-26T21:09:52.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.961+0000 7f8b91200640 1 -- 172.21.15.26:0/947922147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b8c145c20 msgr2=0x7f8b8c0d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:52.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.961+0000 7f8b91200640 1 --2- 172.21.15.26:0/947922147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b8c145c20 0x7f8b8c0d06f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c146e40 tx=0x7f8b7c02a1e0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.961+0000 7f8b91200640 1 -- 172.21.15.26:0/947922147 shutdown_connections 2024-04-26T21:09:52.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.961+0000 7f8b91200640 1 --2- 172.21.15.26:0/947922147 >> v2:172.21.15.86:3300/0 conn(0x7f8b8c1468b0 0x7f8b8c0d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.961+0000 7f8b91200640 1 --2- 172.21.15.26:0/947922147 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8b8c146450 0x7f8b8c0d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.961+0000 7f8b91200640 1 --2- 172.21.15.26:0/947922147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b8c145c20 0x7f8b8c0d06f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.961+0000 7f8b91200640 1 -- 172.21.15.26:0/947922147 >> 172.21.15.26:0/947922147 conn(0x7f8b8c06c690 msgr2=0x7f8b8c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.961+0000 7f8b91200640 1 -- 172.21.15.26:0/947922147 shutdown_connections 2024-04-26T21:09:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.961+0000 7f8b91200640 1 -- 172.21.15.26:0/947922147 wait complete. 2024-04-26T21:09:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.963+0000 7f8b91200640 1 Processor -- start 2024-04-26T21:09:52.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.963+0000 7f8b91200640 1 -- start start 2024-04-26T21:09:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.964+0000 7f8b91200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b8c145c20 0x7f8b8c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.964+0000 7f8b91200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8b8c146450 0x7f8b8c0b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.964+0000 7f8b91200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b8c1468b0 0x7f8b8c1f8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.964+0000 7f8b91200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c146000 con 0x7f8b8c145c20 2024-04-26T21:09:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.964+0000 7f8b91200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c0d01d0 con 0x7f8b8c1468b0 2024-04-26T21:09:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.964+0000 7f8b91200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8b8c15b3d0 con 0x7f8b8c146450 2024-04-26T21:09:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.964+0000 7f8b8b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b8c145c20 0x7f8b8c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.965+0000 7f8b8b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b8c145c20 0x7f8b8c0b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51068/0 (socket says 172.21.15.26:51068) 2024-04-26T21:09:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.965+0000 7f8b8b400640 1 -- 172.21.15.26:0/2319842515 learned_addr learned my addr 172.21.15.26:0/2319842515 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.965+0000 7f8b83e00640 1 --2- 172.21.15.26:0/2319842515 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8b8c146450 0x7f8b8c0b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.965+0000 7f8b8be00640 1 --2- 172.21.15.26:0/2319842515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b8c1468b0 0x7f8b8c1f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.965+0000 7f8b8b400640 1 -- 172.21.15.26:0/2319842515 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8b8c146450 msgr2=0x7f8b8c0b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:52.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.965+0000 7f8b8b400640 1 --2- 172.21.15.26:0/2319842515 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8b8c146450 0x7f8b8c0b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.965+0000 7f8b8b400640 1 -- 172.21.15.26:0/2319842515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b8c1468b0 msgr2=0x7f8b8c1f8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:52.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.965+0000 7f8b8b400640 1 --2- 172.21.15.26:0/2319842515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b8c1468b0 0x7f8b8c1f8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:52.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.965+0000 7f8b8b400640 1 -- 172.21.15.26:0/2319842515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b8c0b5590 con 0x7f8b8c145c20 2024-04-26T21:09:52.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.965+0000 7f8b8be00640 1 --2- 172.21.15.26:0/2319842515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b8c1468b0 0x7f8b8c1f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:52.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.965+0000 7f8b8b400640 1 --2- 172.21.15.26:0/2319842515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b8c145c20 0x7f8b8c0b5050 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c02a220 tx=0x7f8b7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.966+0000 7f8b83400640 1 -- 172.21.15.26:0/2319842515 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c041020 con 0x7f8b8c145c20 2024-04-26T21:09:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.966+0000 7f8b83400640 1 -- 172.21.15.26:0/2319842515 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b7c0306c0 con 0x7f8b8c145c20 2024-04-26T21:09:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.966+0000 7f8b83400640 1 -- 172.21.15.26:0/2319842515 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c037040 con 0x7f8b8c145c20 2024-04-26T21:09:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.966+0000 7f8b91200640 1 -- 172.21.15.26:0/2319842515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b8c1f9170 con 0x7f8b8c145c20 2024-04-26T21:09:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.966+0000 7f8b91200640 1 -- 172.21.15.26:0/2319842515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b8c15b670 con 0x7f8b8c145c20 2024-04-26T21:09:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.967+0000 7f8b91200640 1 -- 172.21.15.26:0/2319842515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b8c1f9690 con 0x7f8b8c145c20 2024-04-26T21:09:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.968+0000 7f8b83400640 1 -- 172.21.15.26:0/2319842515 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b7c044020 con 0x7f8b8c145c20 2024-04-26T21:09:52.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.969+0000 7f8b83400640 1 --2- 172.21.15.26:0/2319842515 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8b6806c8f0 0x7f8b6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:52.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.969+0000 7f8b83400640 1 -- 172.21.15.26:0/2319842515 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f8b7c0ac440 con 0x7f8b8c145c20 2024-04-26T21:09:52.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.969+0000 7f8b83e00640 1 --2- 172.21.15.26:0/2319842515 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8b6806c8f0 0x7f8b6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:52.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.970+0000 7f8b83e00640 1 --2- 172.21.15.26:0/2319842515 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8b6806c8f0 0x7f8b6806edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c13dae0 tx=0x7f8b74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:52.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:52.974+0000 7f8b83400640 1 -- 172.21.15.26:0/2319842515 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8b8c1f9690 con 0x7f8b8c145c20 2024-04-26T21:09:53.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.181+0000 7f8b91200640 1 -- 172.21.15.26:0/2319842515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b8c06a700 con 0x7f8b8c145c20 2024-04-26T21:09:53.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.183+0000 7f8b83400640 1 -- 172.21.15.26:0/2319842515 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16621 (secure 0 0 0) 0x7f8b7c04b030 con 0x7f8b8c145c20 2024-04-26T21:09:53.185 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:53.185 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":34,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:09:50.180593+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:46.646256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:47.613883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:09:53.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.187+0000 7f8b91200640 1 -- 172.21.15.26:0/2319842515 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8b6806c8f0 msgr2=0x7f8b6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:53.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.187+0000 7f8b91200640 1 --2- 172.21.15.26:0/2319842515 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8b6806c8f0 0x7f8b6806edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c13dae0 tx=0x7f8b74008040 comp rx=0 tx=0).stop 2024-04-26T21:09:53.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.187+0000 7f8b91200640 1 -- 172.21.15.26:0/2319842515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b8c145c20 msgr2=0x7f8b8c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:53.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.187+0000 7f8b91200640 1 --2- 172.21.15.26:0/2319842515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b8c145c20 0x7f8b8c0b5050 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c02a220 tx=0x7f8b7c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:09:53.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.187+0000 7f8b91200640 1 -- 172.21.15.26:0/2319842515 shutdown_connections 2024-04-26T21:09:53.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.187+0000 7f8b91200640 1 --2- 172.21.15.26:0/2319842515 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8b6806c8f0 0x7f8b6806edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:53.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.187+0000 7f8b91200640 1 --2- 172.21.15.26:0/2319842515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b8c1468b0 0x7f8b8c1f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:53.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.187+0000 7f8b91200640 1 --2- 172.21.15.26:0/2319842515 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8b8c146450 0x7f8b8c0b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:53.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.187+0000 7f8b91200640 1 --2- 172.21.15.26:0/2319842515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b8c145c20 0x7f8b8c0b5050 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:53.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.187+0000 7f8b91200640 1 -- 172.21.15.26:0/2319842515 >> 172.21.15.26:0/2319842515 conn(0x7f8b8c06c690 msgr2=0x7f8b8c0b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:53.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.187+0000 7f8b91200640 1 -- 172.21.15.26:0/2319842515 shutdown_connections 2024-04-26T21:09:53.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.187+0000 7f8b91200640 1 -- 172.21.15.26:0/2319842515 wait complete. 2024-04-26T21:09:53.207 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:09:53.207 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:09:53.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.412+0000 7fcdda400640 1 Processor -- start 2024-04-26T21:09:53.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.413+0000 7fcdda400640 1 -- start start 2024-04-26T21:09:53.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.414+0000 7fcdda400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcdd40d1c80 0x7fcdd40d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:53.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.414+0000 7fcdda400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcdd40d2060 0x7fcdd4162490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:53.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.414+0000 7fcdda400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdd4162e60 con 0x7fcdd41629d0 2024-04-26T21:09:53.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.414+0000 7fcdda400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fcdd40b5f70 con 0x7fcdd40d2060 2024-04-26T21:09:53.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.414+0000 7fcdda400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcdd40d3830 con 0x7fcdd40d1c80 2024-04-26T21:09:53.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.415+0000 7fcdd3e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcdd40d2060 0x7fcdd4162490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:53.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.415+0000 7fcdd3e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcdd40d2060 0x7fcdd4162490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51078/0 (socket says 172.21.15.26:51078) 2024-04-26T21:09:53.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.415+0000 7fcdd3e00640 1 -- 172.21.15.26:0/2488265323 learned_addr learned my addr 172.21.15.26:0/2488265323 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:53.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.415+0000 7fcdd9000640 1 --2- 172.21.15.26:0/2488265323 >> v2:172.21.15.86:3300/0 conn(0x7fcdd40d1c80 0x7fcdd40d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:53.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.415+0000 7fcdd3e00640 1 -- 172.21.15.26:0/2488265323 >> v2:172.21.15.86:3300/0 conn(0x7fcdd40d1c80 msgr2=0x7fcdd40d3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:53.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.415+0000 7fcdd3e00640 1 --2- 172.21.15.26:0/2488265323 >> v2:172.21.15.86:3300/0 conn(0x7fcdd40d1c80 0x7fcdd40d3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:53.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.415+0000 7fcdd3e00640 1 -- 172.21.15.26:0/2488265323 >> v1:172.21.15.86:6789/0 conn(0x7fcdd41629d0 legacy=0x7fcdd40c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:53.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.415+0000 7fcdd3e00640 1 -- 172.21.15.26:0/2488265323 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd40b61c0 con 0x7fcdd40d2060 2024-04-26T21:09:53.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.416+0000 7fcdd3e00640 1 --2- 172.21.15.26:0/2488265323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdd40d2060 0x7fcdd4162490 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcdd408d3b0 tx=0x7fcdc002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d916ca2b6a0e223e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:53.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.417+0000 7fcdd3400640 1 -- 172.21.15.26:0/2488265323 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0037070 con 0x7fcdd40d2060 2024-04-26T21:09:53.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.417+0000 7fcdd3400640 1 -- 172.21.15.26:0/2488265323 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdc0032440 con 0x7fcdd40d2060 2024-04-26T21:09:53.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.417+0000 7fcdd3400640 1 -- 172.21.15.26:0/2488265323 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0032810 con 0x7fcdd40d2060 2024-04-26T21:09:53.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.417+0000 7fcdda400640 1 -- 172.21.15.26:0/2488265323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdd40d2060 msgr2=0x7fcdd4162490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:53.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.417+0000 7fcdda400640 1 --2- 172.21.15.26:0/2488265323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdd40d2060 0x7fcdd4162490 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcdd408d3b0 tx=0x7fcdc002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:09:53.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.417+0000 7fcdda400640 1 -- 172.21.15.26:0/2488265323 shutdown_connections 2024-04-26T21:09:53.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.418+0000 7fcdda400640 1 --2- 172.21.15.26:0/2488265323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdd40d2060 0x7fcdd4162490 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:53.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.418+0000 7fcdda400640 1 --2- 172.21.15.26:0/2488265323 >> v2:172.21.15.86:3300/0 conn(0x7fcdd40d1c80 0x7fcdd40d3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:53.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.418+0000 7fcdda400640 1 -- 172.21.15.26:0/2488265323 >> 172.21.15.26:0/2488265323 conn(0x7fcdd40d5020 msgr2=0x7fcdd40d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:53.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.418+0000 7fcdda400640 1 -- 172.21.15.26:0/2488265323 shutdown_connections 2024-04-26T21:09:53.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.418+0000 7fcdda400640 1 -- 172.21.15.26:0/2488265323 wait complete. 2024-04-26T21:09:53.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.419+0000 7fcdda400640 1 Processor -- start 2024-04-26T21:09:53.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.419+0000 7fcdda400640 1 -- start start 2024-04-26T21:09:53.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.420+0000 7fcdda400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdd40d1c80 0x7fcdd40ba1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:53.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.420+0000 7fcdda400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdd40d2060 0x7fcdd40cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:53.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.420+0000 7fcdda400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdd41629d0 0x7fcdd40cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:53.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.420+0000 7fcdda400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd4074790 con 0x7fcdd40d2060 2024-04-26T21:09:53.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.420+0000 7fcdda400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd4074610 con 0x7fcdd40d1c80 2024-04-26T21:09:53.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.420+0000 7fcdda400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd40d5b40 con 0x7fcdd41629d0 2024-04-26T21:09:53.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.421+0000 7fcdd3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdd40d2060 0x7fcdd40cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:53.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.421+0000 7fcdd3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdd40d2060 0x7fcdd40cc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51086/0 (socket says 172.21.15.26:51086) 2024-04-26T21:09:53.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.421+0000 7fcdd3e00640 1 -- 172.21.15.26:0/1199394303 learned_addr learned my addr 172.21.15.26:0/1199394303 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:53.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.421+0000 7fcdd3e00640 1 -- 172.21.15.26:0/1199394303 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdd41629d0 msgr2=0x7fcdd40cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:53.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.421+0000 7fcdd3e00640 1 --2- 172.21.15.26:0/1199394303 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdd41629d0 0x7fcdd40cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:53.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.421+0000 7fcdd3e00640 1 -- 172.21.15.26:0/1199394303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdd40d1c80 msgr2=0x7fcdd40ba1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:53.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.421+0000 7fcdd3e00640 1 --2- 172.21.15.26:0/1199394303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdd40d1c80 0x7fcdd40ba1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:53.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.421+0000 7fcdd3e00640 1 -- 172.21.15.26:0/1199394303 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd40b9100 con 0x7fcdd40d2060 2024-04-26T21:09:53.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.421+0000 7fcdd3e00640 1 --2- 172.21.15.26:0/1199394303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdd40d2060 0x7fcdd40cc0f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcdc002a1e0 tx=0x7fcdc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:53.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.422+0000 7fcdd0c00640 1 -- 172.21.15.26:0/1199394303 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0041020 con 0x7fcdd40d2060 2024-04-26T21:09:53.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.422+0000 7fcdd0c00640 1 -- 172.21.15.26:0/1199394303 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdc00306c0 con 0x7fcdd40d2060 2024-04-26T21:09:53.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.422+0000 7fcdda400640 1 -- 172.21.15.26:0/1199394303 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd40b93e0 con 0x7fcdd40d2060 2024-04-26T21:09:53.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.422+0000 7fcdd0c00640 1 -- 172.21.15.26:0/1199394303 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0037040 con 0x7fcdd40d2060 2024-04-26T21:09:53.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.422+0000 7fcdda400640 1 -- 172.21.15.26:0/1199394303 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd40b99a0 con 0x7fcdd40d2060 2024-04-26T21:09:53.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.423+0000 7fcdda400640 1 -- 172.21.15.26:0/1199394303 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdd406a700 con 0x7fcdd40d2060 2024-04-26T21:09:53.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.424+0000 7fcdd0c00640 1 -- 172.21.15.26:0/1199394303 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcdc0044020 con 0x7fcdd40d2060 2024-04-26T21:09:53.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.425+0000 7fcdd0c00640 1 --2- 172.21.15.26:0/1199394303 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcdb406c8f0 0x7fcdb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:53.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.425+0000 7fcdd0c00640 1 -- 172.21.15.26:0/1199394303 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fcdc00ac400 con 0x7fcdd40d2060 2024-04-26T21:09:53.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.425+0000 7fcdd9000640 1 --2- 172.21.15.26:0/1199394303 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcdb406c8f0 0x7fcdb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:53.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.426+0000 7fcdd9000640 1 --2- 172.21.15.26:0/1199394303 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcdb406c8f0 0x7fcdb406edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4006fd0 tx=0x7fcdc4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:53.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.430+0000 7fcdd0c00640 1 -- 172.21.15.26:0/1199394303 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcdc004b030 con 0x7fcdd40d2060 2024-04-26T21:09:53.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.629+0000 7fcdda400640 1 -- 172.21.15.26:0/1199394303 --> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcdd406d010 con 0x7fcdb406c8f0 2024-04-26T21:09:53.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.643+0000 7fcdd0c00640 1 -- 172.21.15.26:0/1199394303 <== mgr.4104 v2:172.21.15.26:6832/3130430358 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251920 (secure 0 0 0) 0x7fcdd406d010 con 0x7fcdb406c8f0 2024-04-26T21:09:53.647 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:53.648 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-26T21:09:53.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.648+0000 7fcdda400640 1 -- 172.21.15.26:0/1199394303 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcdb406c8f0 msgr2=0x7fcdb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:53.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.648+0000 7fcdda400640 1 --2- 172.21.15.26:0/1199394303 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcdb406c8f0 0x7fcdb406edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4006fd0 tx=0x7fcdc4008040 comp rx=0 tx=0).stop 2024-04-26T21:09:53.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.648+0000 7fcdda400640 1 -- 172.21.15.26:0/1199394303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdd40d2060 msgr2=0x7fcdd40cc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:53.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.648+0000 7fcdda400640 1 --2- 172.21.15.26:0/1199394303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdd40d2060 0x7fcdd40cc0f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcdc002a1e0 tx=0x7fcdc00046c0 comp rx=0 tx=0).stop 2024-04-26T21:09:53.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.648+0000 7fcdda400640 1 -- 172.21.15.26:0/1199394303 shutdown_connections 2024-04-26T21:09:53.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.648+0000 7fcdda400640 1 --2- 172.21.15.26:0/1199394303 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcdb406c8f0 0x7fcdb406edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:53.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.648+0000 7fcdda400640 1 --2- 172.21.15.26:0/1199394303 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdd41629d0 0x7fcdd40cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:53.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.648+0000 7fcdda400640 1 --2- 172.21.15.26:0/1199394303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdd40d2060 0x7fcdd40cc0f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:53.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.648+0000 7fcdda400640 1 --2- 172.21.15.26:0/1199394303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdd40d1c80 0x7fcdd40ba1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:53.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.648+0000 7fcdda400640 1 -- 172.21.15.26:0/1199394303 >> 172.21.15.26:0/1199394303 conn(0x7fcdd40d5020 msgr2=0x7fcdd40d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:53.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.648+0000 7fcdda400640 1 -- 172.21.15.26:0/1199394303 shutdown_connections 2024-04-26T21:09:53.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.648+0000 7fcdda400640 1 -- 172.21.15.26:0/1199394303 wait complete. 2024-04-26T21:09:53.667 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-04-26T21:09:53.311120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236944,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737776,"statfs":{"total":773094113280,"available":772851482624,"internally_reserved":0,"allocated":4235264,"data_stored":2345388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001512"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.198737+0000","last_change":"2024-04-26T21:09:49.209518+0000","last_active":"2024-04-26T21:09:50.198737+0000","last_peered":"2024-04-26T21:09:50.198737+0000","last_clean":"2024-04-26T21:09:50.198737+0000","last_became_active":"2024-04-26T21:09:49.207194+0000","last_became_peered":"2024-04-26T21:09:49.207194+0000","last_unstale":"2024-04-26T21:09:50.198737+0000","last_undegraded":"2024-04-26T21:09:50.198737+0000","last_fullsized":"2024-04-26T21:09:50.198737+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:01.353442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.215396+0000","last_change":"2024-04-26T21:09:49.207992+0000","last_active":"2024-04-26T21:09:49.215396+0000","last_peered":"2024-04-26T21:09:49.215396+0000","last_clean":"2024-04-26T21:09:49.215396+0000","last_became_active":"2024-04-26T21:09:49.207596+0000","last_became_peered":"2024-04-26T21:09:49.207596+0000","last_unstale":"2024-04-26T21:09:49.215396+0000","last_undegraded":"2024-04-26T21:09:49.215396+0000","last_fullsized":"2024-04-26T21:09:49.215396+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:27.951387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.212736+0000","last_change":"2024-04-26T21:09:49.211340+0000","last_active":"2024-04-26T21:09:49.212736+0000","last_peered":"2024-04-26T21:09:49.212736+0000","last_clean":"2024-04-26T21:09:49.212736+0000","last_became_active":"2024-04-26T21:09:49.210123+0000","last_became_peered":"2024-04-26T21:09:49.210123+0000","last_unstale":"2024-04-26T21:09:49.212736+0000","last_undegraded":"2024-04-26T21:09:49.212736+0000","last_fullsized":"2024-04-26T21:09:49.212736+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:27.951387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.223908+0000","last_change":"2024-04-26T21:09:49.191678+0000","last_active":"2024-04-26T21:09:50.223908+0000","last_peered":"2024-04-26T21:09:50.223908+0000","last_clean":"2024-04-26T21:09:50.223908+0000","last_became_active":"2024-04-26T21:09:49.191361+0000","last_became_peered":"2024-04-26T21:09:49.191361+0000","last_unstale":"2024-04-26T21:09:50.223908+0000","last_undegraded":"2024-04-26T21:09:50.223908+0000","last_fullsized":"2024-04-26T21:09:50.223908+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:27.951387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.210297+0000","last_change":"2024-04-26T21:09:49.207043+0000","last_active":"2024-04-26T21:09:49.210297+0000","last_peered":"2024-04-26T21:09:49.210297+0000","last_clean":"2024-04-26T21:09:49.210297+0000","last_became_active":"2024-04-26T21:09:49.206755+0000","last_became_peered":"2024-04-26T21:09:49.206755+0000","last_unstale":"2024-04-26T21:09:49.210297+0000","last_undegraded":"2024-04-26T21:09:49.210297+0000","last_fullsized":"2024-04-26T21:09:49.210297+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:22.050808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.216617+0000","last_change":"2024-04-26T21:09:49.210589+0000","last_active":"2024-04-26T21:09:49.216617+0000","last_peered":"2024-04-26T21:09:49.216617+0000","last_clean":"2024-04-26T21:09:49.216617+0000","last_became_active":"2024-04-26T21:09:49.210194+0000","last_became_peered":"2024-04-26T21:09:49.210194+0000","last_unstale":"2024-04-26T21:09:49.216617+0000","last_undegraded":"2024-04-26T21:09:49.216617+0000","last_fullsized":"2024-04-26T21:09:49.216617+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:37:42.004263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.196845+0000","last_change":"2024-04-26T21:09:49.204070+0000","last_active":"2024-04-26T21:09:50.196845+0000","last_peered":"2024-04-26T21:09:50.196845+0000","last_clean":"2024-04-26T21:09:50.196845+0000","last_became_active":"2024-04-26T21:09:49.202741+0000","last_became_peered":"2024-04-26T21:09:49.202741+0000","last_unstale":"2024-04-26T21:09:50.196845+0000","last_undegraded":"2024-04-26T21:09:50.196845+0000","last_fullsized":"2024-04-26T21:09:50.196845+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:37:42.004263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.196754+0000","last_change":"2024-04-26T21:09:49.205910+0000","last_active":"2024-04-26T21:09:50.196754+0000","last_peered":"2024-04-26T21:09:50.196754+0000","last_clean":"2024-04-26T21:09:50.196754+0000","last_became_active":"2024-04-26T21:09:49.205620+0000","last_became_peered":"2024-04-26T21:09:49.205620+0000","last_unstale":"2024-04-26T21:09:50.196754+0000","last_undegraded":"2024-04-26T21:09:50.196754+0000","last_fullsized":"2024-04-26T21:09:50.196754+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:53.758778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.248215+0000","last_change":"2024-04-26T21:09:49.188874+0000","last_active":"2024-04-26T21:09:50.248215+0000","last_peered":"2024-04-26T21:09:50.248215+0000","last_clean":"2024-04-26T21:09:50.248215+0000","last_became_active":"2024-04-26T21:09:49.187596+0000","last_became_peered":"2024-04-26T21:09:49.187596+0000","last_unstale":"2024-04-26T21:09:50.248215+0000","last_undegraded":"2024-04-26T21:09:50.248215+0000","last_fullsized":"2024-04-26T21:09:50.248215+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:27.951387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.224346+0000","last_change":"2024-04-26T21:09:49.211984+0000","last_active":"2024-04-26T21:09:50.224346+0000","last_peered":"2024-04-26T21:09:50.224346+0000","last_clean":"2024-04-26T21:09:50.224346+0000","last_became_active":"2024-04-26T21:09:49.211637+0000","last_became_peered":"2024-04-26T21:09:49.211637+0000","last_unstale":"2024-04-26T21:09:50.224346+0000","last_undegraded":"2024-04-26T21:09:50.224346+0000","last_fullsized":"2024-04-26T21:09:50.224346+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:35.597618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.197962+0000","last_change":"2024-04-26T21:09:49.217009+0000","last_active":"2024-04-26T21:09:50.197962+0000","last_peered":"2024-04-26T21:09:50.197962+0000","last_clean":"2024-04-26T21:09:50.197962+0000","last_became_active":"2024-04-26T21:09:49.216635+0000","last_became_peered":"2024-04-26T21:09:49.216635+0000","last_unstale":"2024-04-26T21:09:50.197962+0000","last_undegraded":"2024-04-26T21:09:50.197962+0000","last_fullsized":"2024-04-26T21:09:50.197962+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:58.695182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.207580+0000","last_change":"2024-04-26T21:09:49.202752+0000","last_active":"2024-04-26T21:09:49.207580+0000","last_peered":"2024-04-26T21:09:49.207580+0000","last_clean":"2024-04-26T21:09:49.207580+0000","last_became_active":"2024-04-26T21:09:49.202103+0000","last_became_peered":"2024-04-26T21:09:49.202103+0000","last_unstale":"2024-04-26T21:09:49.207580+0000","last_undegraded":"2024-04-26T21:09:49.207580+0000","last_fullsized":"2024-04-26T21:09:49.207580+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:36.246542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.188106+0000","last_change":"2024-04-26T21:09:49.186462+0000","last_active":"2024-04-26T21:09:49.188106+0000","last_peered":"2024-04-26T21:09:49.188106+0000","last_clean":"2024-04-26T21:09:49.188106+0000","last_became_active":"2024-04-26T21:09:49.185938+0000","last_became_peered":"2024-04-26T21:09:49.185938+0000","last_unstale":"2024-04-26T21:09:49.188106+0000","last_undegraded":"2024-04-26T21:09:49.188106+0000","last_fullsized":"2024-04-26T21:09:49.188106+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:45.675322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.212215+0000","last_change":"2024-04-26T21:09:49.203532+0000","last_active":"2024-04-26T21:09:49.212215+0000","last_peered":"2024-04-26T21:09:49.212215+0000","last_clean":"2024-04-26T21:09:49.212215+0000","last_became_active":"2024-04-26T21:09:49.203232+0000","last_became_peered":"2024-04-26T21:09:49.203232+0000","last_unstale":"2024-04-26T21:09:49.212215+0000","last_undegraded":"2024-04-26T21:09:49.212215+0000","last_fullsized":"2024-04-26T21:09:49.212215+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:18.403639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.207531+0000","last_change":"2024-04-26T21:09:49.203170+0000","last_active":"2024-04-26T21:09:49.207531+0000","last_peered":"2024-04-26T21:09:49.207531+0000","last_clean":"2024-04-26T21:09:49.207531+0000","last_became_active":"2024-04-26T21:09:49.202878+0000","last_became_peered":"2024-04-26T21:09:49.202878+0000","last_unstale":"2024-04-26T21:09:49.207531+0000","last_undegraded":"2024-04-26T21:09:49.207531+0000","last_fullsized":"2024-04-26T21:09:49.207531+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:29.134501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.197962+0000","last_change":"2024-04-26T21:09:49.217585+0000","last_active":"2024-04-26T21:09:50.197962+0000","last_peered":"2024-04-26T21:09:50.197962+0000","last_clean":"2024-04-26T21:09:50.197962+0000","last_became_active":"2024-04-26T21:09:49.217286+0000","last_became_peered":"2024-04-26T21:09:49.217286+0000","last_unstale":"2024-04-26T21:09:50.197962+0000","last_undegraded":"2024-04-26T21:09:50.197962+0000","last_fullsized":"2024-04-26T21:09:50.197962+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:20.043920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.198987+0000","last_change":"2024-04-26T21:09:49.207303+0000","last_active":"2024-04-26T21:09:50.198987+0000","last_peered":"2024-04-26T21:09:50.198987+0000","last_clean":"2024-04-26T21:09:50.198987+0000","last_became_active":"2024-04-26T21:09:49.207016+0000","last_became_peered":"2024-04-26T21:09:49.207016+0000","last_unstale":"2024-04-26T21:09:50.198987+0000","last_undegraded":"2024-04-26T21:09:50.198987+0000","last_fullsized":"2024-04-26T21:09:50.198987+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:27.020554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.218189+0000","last_change":"2024-04-26T21:09:49.215174+0000","last_active":"2024-04-26T21:09:49.218189+0000","last_peered":"2024-04-26T21:09:49.218189+0000","last_clean":"2024-04-26T21:09:49.218189+0000","last_became_active":"2024-04-26T21:09:49.213199+0000","last_became_peered":"2024-04-26T21:09:49.213199+0000","last_unstale":"2024-04-26T21:09:49.218189+0000","last_undegraded":"2024-04-26T21:09:49.218189+0000","last_fullsized":"2024-04-26T21:09:49.218189+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:51.306403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.216737+0000","last_change":"2024-04-26T21:09:49.212245+0000","last_active":"2024-04-26T21:09:49.216737+0000","last_peered":"2024-04-26T21:09:49.216737+0000","last_clean":"2024-04-26T21:09:49.216737+0000","last_became_active":"2024-04-26T21:09:49.211925+0000","last_became_peered":"2024-04-26T21:09:49.211925+0000","last_unstale":"2024-04-26T21:09:49.216737+0000","last_undegraded":"2024-04-26T21:09:49.216737+0000","last_fullsized":"2024-04-26T21:09:49.216737+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:35.597618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.208340+0000","last_change":"2024-04-26T21:09:49.204645+0000","last_active":"2024-04-26T21:09:49.208340+0000","last_peered":"2024-04-26T21:09:49.208340+0000","last_clean":"2024-04-26T21:09:49.208340+0000","last_became_active":"2024-04-26T21:09:49.204326+0000","last_became_peered":"2024-04-26T21:09:49.204326+0000","last_unstale":"2024-04-26T21:09:49.208340+0000","last_undegraded":"2024-04-26T21:09:49.208340+0000","last_fullsized":"2024-04-26T21:09:49.208340+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:09.058773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.247656+0000","last_change":"2024-04-26T21:09:49.194892+0000","last_active":"2024-04-26T21:09:50.247656+0000","last_peered":"2024-04-26T21:09:50.247656+0000","last_clean":"2024-04-26T21:09:50.247656+0000","last_became_active":"2024-04-26T21:09:49.194359+0000","last_became_peered":"2024-04-26T21:09:49.194359+0000","last_unstale":"2024-04-26T21:09:50.247656+0000","last_undegraded":"2024-04-26T21:09:50.247656+0000","last_fullsized":"2024-04-26T21:09:50.247656+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:18.880151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.247663+0000","last_change":"2024-04-26T21:09:49.201279+0000","last_active":"2024-04-26T21:09:50.247663+0000","last_peered":"2024-04-26T21:09:50.247663+0000","last_clean":"2024-04-26T21:09:50.247663+0000","last_became_active":"2024-04-26T21:09:49.200954+0000","last_became_peered":"2024-04-26T21:09:49.200954+0000","last_unstale":"2024-04-26T21:09:50.247663+0000","last_undegraded":"2024-04-26T21:09:50.247663+0000","last_fullsized":"2024-04-26T21:09:50.247663+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:07:20.080838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.192766+0000","last_change":"2024-04-26T21:09:49.194084+0000","last_active":"2024-04-26T21:09:50.192766+0000","last_peered":"2024-04-26T21:09:50.192766+0000","last_clean":"2024-04-26T21:09:50.192766+0000","last_became_active":"2024-04-26T21:09:49.193781+0000","last_became_peered":"2024-04-26T21:09:49.193781+0000","last_unstale":"2024-04-26T21:09:50.192766+0000","last_undegraded":"2024-04-26T21:09:50.192766+0000","last_fullsized":"2024-04-26T21:09:50.192766+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:29.134501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.209144+0000","last_change":"2024-04-26T21:09:49.205033+0000","last_active":"2024-04-26T21:09:49.209144+0000","last_peered":"2024-04-26T21:09:49.209144+0000","last_clean":"2024-04-26T21:09:49.209144+0000","last_became_active":"2024-04-26T21:09:49.204717+0000","last_became_peered":"2024-04-26T21:09:49.204717+0000","last_unstale":"2024-04-26T21:09:49.209144+0000","last_undegraded":"2024-04-26T21:09:49.209144+0000","last_fullsized":"2024-04-26T21:09:49.209144+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:19.183103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.224599+0000","last_change":"2024-04-26T21:09:49.209351+0000","last_active":"2024-04-26T21:09:50.224599+0000","last_peered":"2024-04-26T21:09:50.224599+0000","last_clean":"2024-04-26T21:09:50.224599+0000","last_became_active":"2024-04-26T21:09:49.209042+0000","last_became_peered":"2024-04-26T21:09:49.209042+0000","last_unstale":"2024-04-26T21:09:50.224599+0000","last_undegraded":"2024-04-26T21:09:50.224599+0000","last_fullsized":"2024-04-26T21:09:50.224599+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:27.737169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.215375+0000","last_change":"2024-04-26T21:09:49.203091+0000","last_active":"2024-04-26T21:09:49.215375+0000","last_peered":"2024-04-26T21:09:49.215375+0000","last_clean":"2024-04-26T21:09:49.215375+0000","last_became_active":"2024-04-26T21:09:49.200996+0000","last_became_peered":"2024-04-26T21:09:49.200996+0000","last_unstale":"2024-04-26T21:09:49.215375+0000","last_undegraded":"2024-04-26T21:09:49.215375+0000","last_fullsized":"2024-04-26T21:09:49.215375+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:07:20.080838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.224383+0000","last_change":"2024-04-26T21:09:49.214882+0000","last_active":"2024-04-26T21:09:50.224383+0000","last_peered":"2024-04-26T21:09:50.224383+0000","last_clean":"2024-04-26T21:09:50.224383+0000","last_became_active":"2024-04-26T21:09:49.214023+0000","last_became_peered":"2024-04-26T21:09:49.214023+0000","last_unstale":"2024-04-26T21:09:50.224383+0000","last_undegraded":"2024-04-26T21:09:50.224383+0000","last_fullsized":"2024-04-26T21:09:50.224383+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:51.306403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.206386+0000","last_change":"2024-04-26T21:09:49.201229+0000","last_active":"2024-04-26T21:09:49.206386+0000","last_peered":"2024-04-26T21:09:49.206386+0000","last_clean":"2024-04-26T21:09:49.206386+0000","last_became_active":"2024-04-26T21:09:49.200412+0000","last_became_peered":"2024-04-26T21:09:49.200412+0000","last_unstale":"2024-04-26T21:09:49.206386+0000","last_undegraded":"2024-04-26T21:09:49.206386+0000","last_fullsized":"2024-04-26T21:09:49.206386+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:01.353442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.248182+0000","last_change":"2024-04-26T21:09:49.195095+0000","last_active":"2024-04-26T21:09:50.248182+0000","last_peered":"2024-04-26T21:09:50.248182+0000","last_clean":"2024-04-26T21:09:50.248182+0000","last_became_active":"2024-04-26T21:09:49.194783+0000","last_became_peered":"2024-04-26T21:09:49.194783+0000","last_unstale":"2024-04-26T21:09:50.248182+0000","last_undegraded":"2024-04-26T21:09:50.248182+0000","last_fullsized":"2024-04-26T21:09:50.248182+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:18.403639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.196904+0000","last_change":"2024-04-26T21:09:49.205089+0000","last_active":"2024-04-26T21:09:50.196904+0000","last_peered":"2024-04-26T21:09:50.196904+0000","last_clean":"2024-04-26T21:09:50.196904+0000","last_became_active":"2024-04-26T21:09:49.204713+0000","last_became_peered":"2024-04-26T21:09:49.204713+0000","last_unstale":"2024-04-26T21:09:50.196904+0000","last_undegraded":"2024-04-26T21:09:50.196904+0000","last_fullsized":"2024-04-26T21:09:50.196904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:35.597618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.197603+0000","last_change":"2024-04-26T21:09:49.196401+0000","last_active":"2024-04-26T21:09:50.197603+0000","last_peered":"2024-04-26T21:09:50.197603+0000","last_clean":"2024-04-26T21:09:50.197603+0000","last_became_active":"2024-04-26T21:09:49.193990+0000","last_became_peered":"2024-04-26T21:09:49.193990+0000","last_unstale":"2024-04-26T21:09:50.197603+0000","last_undegraded":"2024-04-26T21:09:50.197603+0000","last_fullsized":"2024-04-26T21:09:50.197603+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:17.572409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.197147+0000","last_change":"2024-04-26T21:09:49.199637+0000","last_active":"2024-04-26T21:09:50.197147+0000","last_peered":"2024-04-26T21:09:50.197147+0000","last_clean":"2024-04-26T21:09:50.197147+0000","last_became_active":"2024-04-26T21:09:49.199067+0000","last_became_peered":"2024-04-26T21:09:49.199067+0000","last_unstale":"2024-04-26T21:09:50.197147+0000","last_undegraded":"2024-04-26T21:09:50.197147+0000","last_fullsized":"2024-04-26T21:09:50.197147+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:27.737169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.193227+0000","last_change":"2024-04-26T21:09:49.196346+0000","last_active":"2024-04-26T21:09:50.193227+0000","last_peered":"2024-04-26T21:09:50.193227+0000","last_clean":"2024-04-26T21:09:50.193227+0000","last_became_active":"2024-04-26T21:09:49.196123+0000","last_became_peered":"2024-04-26T21:09:49.196123+0000","last_unstale":"2024-04-26T21:09:50.193227+0000","last_undegraded":"2024-04-26T21:09:50.193227+0000","last_fullsized":"2024-04-26T21:09:50.193227+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:18.880151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.199198+0000","last_change":"2024-04-26T21:09:48.178859+0000","last_active":"2024-04-26T21:09:50.199198+0000","last_peered":"2024-04-26T21:09:50.199198+0000","last_clean":"2024-04-26T21:09:50.199198+0000","last_became_active":"2024-04-26T21:09:48.178328+0000","last_became_peered":"2024-04-26T21:09:48.178328+0000","last_unstale":"2024-04-26T21:09:50.199198+0000","last_undegraded":"2024-04-26T21:09:50.199198+0000","last_fullsized":"2024-04-26T21:09:50.199198+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:11.900171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.198926+0000","last_change":"2024-04-26T21:09:49.199164+0000","last_active":"2024-04-26T21:09:50.198926+0000","last_peered":"2024-04-26T21:09:50.198926+0000","last_clean":"2024-04-26T21:09:50.198926+0000","last_became_active":"2024-04-26T21:09:49.198852+0000","last_became_peered":"2024-04-26T21:09:49.198852+0000","last_unstale":"2024-04-26T21:09:50.198926+0000","last_undegraded":"2024-04-26T21:09:50.198926+0000","last_fullsized":"2024-04-26T21:09:50.198926+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:29.134501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.193186+0000","last_change":"2024-04-26T21:09:48.174461+0000","last_active":"2024-04-26T21:09:50.193186+0000","last_peered":"2024-04-26T21:09:50.193186+0000","last_clean":"2024-04-26T21:09:50.193186+0000","last_became_active":"2024-04-26T21:09:48.173832+0000","last_became_peered":"2024-04-26T21:09:48.173832+0000","last_unstale":"2024-04-26T21:09:50.193186+0000","last_undegraded":"2024-04-26T21:09:50.193186+0000","last_fullsized":"2024-04-26T21:09:50.193186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:04:02.163653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.248364+0000","last_change":"2024-04-26T21:09:49.201270+0000","last_active":"2024-04-26T21:09:50.248364+0000","last_peered":"2024-04-26T21:09:50.248364+0000","last_clean":"2024-04-26T21:09:50.248364+0000","last_became_active":"2024-04-26T21:09:49.200958+0000","last_became_peered":"2024-04-26T21:09:49.200958+0000","last_unstale":"2024-04-26T21:09:50.248364+0000","last_undegraded":"2024-04-26T21:09:50.248364+0000","last_fullsized":"2024-04-26T21:09:50.248364+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:53.758778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.230685+0000","last_change":"2024-04-26T21:09:48.162213+0000","last_active":"2024-04-26T21:09:49.230685+0000","last_peered":"2024-04-26T21:09:49.230685+0000","last_clean":"2024-04-26T21:09:49.230685+0000","last_became_active":"2024-04-26T21:09:48.161912+0000","last_became_peered":"2024-04-26T21:09:48.161912+0000","last_unstale":"2024-04-26T21:09:49.230685+0000","last_undegraded":"2024-04-26T21:09:49.230685+0000","last_fullsized":"2024-04-26T21:09:49.230685+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:20.091000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.224295+0000","last_change":"2024-04-26T21:09:49.198708+0000","last_active":"2024-04-26T21:09:50.224295+0000","last_peered":"2024-04-26T21:09:50.224295+0000","last_clean":"2024-04-26T21:09:50.224295+0000","last_became_active":"2024-04-26T21:09:49.192637+0000","last_became_peered":"2024-04-26T21:09:49.192637+0000","last_unstale":"2024-04-26T21:09:50.224295+0000","last_undegraded":"2024-04-26T21:09:50.224295+0000","last_fullsized":"2024-04-26T21:09:50.224295+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:37:42.004263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.226886+0000","last_change":"2024-04-26T21:09:48.166374+0000","last_active":"2024-04-26T21:09:49.226886+0000","last_peered":"2024-04-26T21:09:49.226886+0000","last_clean":"2024-04-26T21:09:49.226886+0000","last_became_active":"2024-04-26T21:09:48.166078+0000","last_became_peered":"2024-04-26T21:09:48.166078+0000","last_unstale":"2024-04-26T21:09:49.226886+0000","last_undegraded":"2024-04-26T21:09:49.226886+0000","last_fullsized":"2024-04-26T21:09:49.226886+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:31:10.713295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.197009+0000","last_change":"2024-04-26T21:09:49.210181+0000","last_active":"2024-04-26T21:09:50.197009+0000","last_peered":"2024-04-26T21:09:50.197009+0000","last_clean":"2024-04-26T21:09:50.197009+0000","last_became_active":"2024-04-26T21:09:49.209628+0000","last_became_peered":"2024-04-26T21:09:49.209628+0000","last_unstale":"2024-04-26T21:09:50.197009+0000","last_undegraded":"2024-04-26T21:09:50.197009+0000","last_fullsized":"2024-04-26T21:09:50.197009+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:51.306403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.248013+0000","last_change":"2024-04-26T21:09:48.157178+0000","last_active":"2024-04-26T21:09:50.248013+0000","last_peered":"2024-04-26T21:09:50.248013+0000","last_clean":"2024-04-26T21:09:50.248013+0000","last_became_active":"2024-04-26T21:09:48.156051+0000","last_became_peered":"2024-04-26T21:09:48.156051+0000","last_unstale":"2024-04-26T21:09:50.248013+0000","last_undegraded":"2024-04-26T21:09:50.248013+0000","last_fullsized":"2024-04-26T21:09:50.248013+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:51:05.346543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.206474+0000","last_change":"2024-04-26T21:09:49.198957+0000","last_active":"2024-04-26T21:09:49.206474+0000","last_peered":"2024-04-26T21:09:49.206474+0000","last_clean":"2024-04-26T21:09:49.206474+0000","last_became_active":"2024-04-26T21:09:49.198644+0000","last_became_peered":"2024-04-26T21:09:49.198644+0000","last_unstale":"2024-04-26T21:09:49.206474+0000","last_undegraded":"2024-04-26T21:09:49.206474+0000","last_fullsized":"2024-04-26T21:09:49.206474+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:31.125270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.247354+0000","last_change":"2024-04-26T21:09:48.161763+0000","last_active":"2024-04-26T21:09:50.247354+0000","last_peered":"2024-04-26T21:09:50.247354+0000","last_clean":"2024-04-26T21:09:50.247354+0000","last_became_active":"2024-04-26T21:09:48.161279+0000","last_became_peered":"2024-04-26T21:09:48.161279+0000","last_unstale":"2024-04-26T21:09:50.247354+0000","last_undegraded":"2024-04-26T21:09:50.247354+0000","last_fullsized":"2024-04-26T21:09:50.247354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:21.035783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.247279+0000","last_change":"2024-04-26T21:09:49.196379+0000","last_active":"2024-04-26T21:09:50.247279+0000","last_peered":"2024-04-26T21:09:50.247279+0000","last_clean":"2024-04-26T21:09:50.247279+0000","last_became_active":"2024-04-26T21:09:49.196087+0000","last_became_peered":"2024-04-26T21:09:49.196087+0000","last_unstale":"2024-04-26T21:09:50.247279+0000","last_undegraded":"2024-04-26T21:09:50.247279+0000","last_fullsized":"2024-04-26T21:09:50.247279+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:29.134501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.188775+0000","last_change":"2024-04-26T21:09:48.145300+0000","last_active":"2024-04-26T21:09:49.188775+0000","last_peered":"2024-04-26T21:09:49.188775+0000","last_clean":"2024-04-26T21:09:49.188775+0000","last_became_active":"2024-04-26T21:09:48.144878+0000","last_became_peered":"2024-04-26T21:09:48.144878+0000","last_unstale":"2024-04-26T21:09:49.188775+0000","last_undegraded":"2024-04-26T21:09:49.188775+0000","last_fullsized":"2024-04-26T21:09:49.188775+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:30.110244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.223630+0000","last_change":"2024-04-26T21:09:49.209773+0000","last_active":"2024-04-26T21:09:50.223630+0000","last_peered":"2024-04-26T21:09:50.223630+0000","last_clean":"2024-04-26T21:09:50.223630+0000","last_became_active":"2024-04-26T21:09:49.209401+0000","last_became_peered":"2024-04-26T21:09:49.209401+0000","last_unstale":"2024-04-26T21:09:50.223630+0000","last_undegraded":"2024-04-26T21:09:50.223630+0000","last_fullsized":"2024-04-26T21:09:50.223630+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:07:20.080838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.247080+0000","last_change":"2024-04-26T21:09:48.165782+0000","last_active":"2024-04-26T21:09:50.247080+0000","last_peered":"2024-04-26T21:09:50.247080+0000","last_clean":"2024-04-26T21:09:50.247080+0000","last_became_active":"2024-04-26T21:09:48.165428+0000","last_became_peered":"2024-04-26T21:09:48.165428+0000","last_unstale":"2024-04-26T21:09:50.247080+0000","last_undegraded":"2024-04-26T21:09:50.247080+0000","last_fullsized":"2024-04-26T21:09:50.247080+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:41.927036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.218163+0000","last_change":"2024-04-26T21:09:49.216343+0000","last_active":"2024-04-26T21:09:49.218163+0000","last_peered":"2024-04-26T21:09:49.218163+0000","last_clean":"2024-04-26T21:09:49.218163+0000","last_became_active":"2024-04-26T21:09:49.216055+0000","last_became_peered":"2024-04-26T21:09:49.216055+0000","last_unstale":"2024-04-26T21:09:49.218163+0000","last_undegraded":"2024-04-26T21:09:49.218163+0000","last_fullsized":"2024-04-26T21:09:49.218163+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:53.758778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.189024+0000","last_change":"2024-04-26T21:09:48.173351+0000","last_active":"2024-04-26T21:09:49.189024+0000","last_peered":"2024-04-26T21:09:49.189024+0000","last_clean":"2024-04-26T21:09:49.189024+0000","last_became_active":"2024-04-26T21:09:48.173109+0000","last_became_peered":"2024-04-26T21:09:48.173109+0000","last_unstale":"2024-04-26T21:09:49.189024+0000","last_undegraded":"2024-04-26T21:09:49.189024+0000","last_fullsized":"2024-04-26T21:09:49.189024+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:35.231516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.212999+0000","last_change":"2024-04-26T21:09:49.208784+0000","last_active":"2024-04-26T21:09:49.212999+0000","last_peered":"2024-04-26T21:09:49.212999+0000","last_clean":"2024-04-26T21:09:49.212999+0000","last_became_active":"2024-04-26T21:09:49.208510+0000","last_became_peered":"2024-04-26T21:09:49.208510+0000","last_unstale":"2024-04-26T21:09:49.212999+0000","last_undegraded":"2024-04-26T21:09:49.212999+0000","last_fullsized":"2024-04-26T21:09:49.212999+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:53.758778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.193312+0000","last_change":"2024-04-26T21:09:48.159974+0000","last_active":"2024-04-26T21:09:50.193312+0000","last_peered":"2024-04-26T21:09:50.193312+0000","last_clean":"2024-04-26T21:09:50.193312+0000","last_became_active":"2024-04-26T21:09:48.159623+0000","last_became_peered":"2024-04-26T21:09:48.159623+0000","last_unstale":"2024-04-26T21:09:50.193312+0000","last_undegraded":"2024-04-26T21:09:50.193312+0000","last_fullsized":"2024-04-26T21:09:50.193312+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:56.909021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.205877+0000","last_change":"2024-04-26T21:09:49.199148+0000","last_active":"2024-04-26T21:09:49.205877+0000","last_peered":"2024-04-26T21:09:49.205877+0000","last_clean":"2024-04-26T21:09:49.205877+0000","last_became_active":"2024-04-26T21:09:49.198829+0000","last_became_peered":"2024-04-26T21:09:49.198829+0000","last_unstale":"2024-04-26T21:09:49.205877+0000","last_undegraded":"2024-04-26T21:09:49.205877+0000","last_fullsized":"2024-04-26T21:09:49.205877+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:19.183103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.229518+0000","last_change":"2024-04-26T21:09:48.178060+0000","last_active":"2024-04-26T21:09:49.229518+0000","last_peered":"2024-04-26T21:09:49.229518+0000","last_clean":"2024-04-26T21:09:49.229518+0000","last_became_active":"2024-04-26T21:09:48.177786+0000","last_became_peered":"2024-04-26T21:09:48.177786+0000","last_unstale":"2024-04-26T21:09:49.229518+0000","last_undegraded":"2024-04-26T21:09:49.229518+0000","last_fullsized":"2024-04-26T21:09:49.229518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:16.557384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.212713+0000","last_change":"2024-04-26T21:09:49.211074+0000","last_active":"2024-04-26T21:09:49.212713+0000","last_peered":"2024-04-26T21:09:49.212713+0000","last_clean":"2024-04-26T21:09:49.212713+0000","last_became_active":"2024-04-26T21:09:49.210724+0000","last_became_peered":"2024-04-26T21:09:49.210724+0000","last_unstale":"2024-04-26T21:09:49.212713+0000","last_undegraded":"2024-04-26T21:09:49.212713+0000","last_fullsized":"2024-04-26T21:09:49.212713+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:18.403639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.234430+0000","last_change":"2024-04-26T21:09:48.165226+0000","last_active":"2024-04-26T21:09:49.234430+0000","last_peered":"2024-04-26T21:09:49.234430+0000","last_clean":"2024-04-26T21:09:49.234430+0000","last_became_active":"2024-04-26T21:09:48.164925+0000","last_became_peered":"2024-04-26T21:09:48.164925+0000","last_unstale":"2024-04-26T21:09:49.234430+0000","last_undegraded":"2024-04-26T21:09:49.234430+0000","last_fullsized":"2024-04-26T21:09:49.234430+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:00.338416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.223703+0000","last_change":"2024-04-26T21:09:49.209883+0000","last_active":"2024-04-26T21:09:50.223703+0000","last_peered":"2024-04-26T21:09:50.223703+0000","last_clean":"2024-04-26T21:09:50.223703+0000","last_became_active":"2024-04-26T21:09:49.209072+0000","last_became_peered":"2024-04-26T21:09:49.209072+0000","last_unstale":"2024-04-26T21:09:50.223703+0000","last_undegraded":"2024-04-26T21:09:50.223703+0000","last_fullsized":"2024-04-26T21:09:50.223703+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:18.403639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.248213+0000","last_change":"2024-04-26T21:09:48.170697+0000","last_active":"2024-04-26T21:09:50.248213+0000","last_peered":"2024-04-26T21:09:50.248213+0000","last_clean":"2024-04-26T21:09:50.248213+0000","last_became_active":"2024-04-26T21:09:48.169662+0000","last_became_peered":"2024-04-26T21:09:48.169662+0000","last_unstale":"2024-04-26T21:09:50.248213+0000","last_undegraded":"2024-04-26T21:09:50.248213+0000","last_fullsized":"2024-04-26T21:09:50.248213+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:39.478970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.207536+0000","last_change":"2024-04-26T21:09:49.200924+0000","last_active":"2024-04-26T21:09:49.207536+0000","last_peered":"2024-04-26T21:09:49.207536+0000","last_clean":"2024-04-26T21:09:49.207536+0000","last_became_active":"2024-04-26T21:09:49.200546+0000","last_became_peered":"2024-04-26T21:09:49.200546+0000","last_unstale":"2024-04-26T21:09:49.207536+0000","last_undegraded":"2024-04-26T21:09:49.207536+0000","last_fullsized":"2024-04-26T21:09:49.207536+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:18.880151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.223798+0000","last_change":"2024-04-26T21:09:48.169726+0000","last_active":"2024-04-26T21:09:50.223798+0000","last_peered":"2024-04-26T21:09:50.223798+0000","last_clean":"2024-04-26T21:09:50.223798+0000","last_became_active":"2024-04-26T21:09:48.169328+0000","last_became_peered":"2024-04-26T21:09:48.169328+0000","last_unstale":"2024-04-26T21:09:50.223798+0000","last_undegraded":"2024-04-26T21:09:50.223798+0000","last_fullsized":"2024-04-26T21:09:50.223798+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:11.900171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.197582+0000","last_change":"2024-04-26T21:09:49.199729+0000","last_active":"2024-04-26T21:09:50.197582+0000","last_peered":"2024-04-26T21:09:50.197582+0000","last_clean":"2024-04-26T21:09:50.197582+0000","last_became_active":"2024-04-26T21:09:49.199395+0000","last_became_peered":"2024-04-26T21:09:49.199395+0000","last_unstale":"2024-04-26T21:09:50.197582+0000","last_undegraded":"2024-04-26T21:09:50.197582+0000","last_fullsized":"2024-04-26T21:09:50.197582+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:46.120808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.188442+0000","last_change":"2024-04-26T21:09:48.172714+0000","last_active":"2024-04-26T21:09:49.188442+0000","last_peered":"2024-04-26T21:09:49.188442+0000","last_clean":"2024-04-26T21:09:49.188442+0000","last_became_active":"2024-04-26T21:09:48.172407+0000","last_became_peered":"2024-04-26T21:09:48.172407+0000","last_unstale":"2024-04-26T21:09:49.188442+0000","last_undegraded":"2024-04-26T21:09:49.188442+0000","last_fullsized":"2024-04-26T21:09:49.188442+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:16.557384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.197677+0000","last_change":"2024-04-26T21:09:49.197339+0000","last_active":"2024-04-26T21:09:50.197677+0000","last_peered":"2024-04-26T21:09:50.197677+0000","last_clean":"2024-04-26T21:09:50.197677+0000","last_became_active":"2024-04-26T21:09:49.197051+0000","last_became_peered":"2024-04-26T21:09:49.197051+0000","last_unstale":"2024-04-26T21:09:50.197677+0000","last_undegraded":"2024-04-26T21:09:50.197677+0000","last_fullsized":"2024-04-26T21:09:50.197677+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:31.125270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.234005+0000","last_change":"2024-04-26T21:09:48.165303+0000","last_active":"2024-04-26T21:09:49.234005+0000","last_peered":"2024-04-26T21:09:49.234005+0000","last_clean":"2024-04-26T21:09:49.234005+0000","last_became_active":"2024-04-26T21:09:48.164809+0000","last_became_peered":"2024-04-26T21:09:48.164809+0000","last_unstale":"2024-04-26T21:09:49.234005+0000","last_undegraded":"2024-04-26T21:09:49.234005+0000","last_fullsized":"2024-04-26T21:09:49.234005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:24:22.909294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.247601+0000","last_change":"2024-04-26T21:09:49.204572+0000","last_active":"2024-04-26T21:09:50.247601+0000","last_peered":"2024-04-26T21:09:50.247601+0000","last_clean":"2024-04-26T21:09:50.247601+0000","last_became_active":"2024-04-26T21:09:49.204183+0000","last_became_peered":"2024-04-26T21:09:49.204183+0000","last_unstale":"2024-04-26T21:09:50.247601+0000","last_undegraded":"2024-04-26T21:09:50.247601+0000","last_fullsized":"2024-04-26T21:09:50.247601+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:19.183103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.224721+0000","last_change":"2024-04-26T21:09:48.164054+0000","last_active":"2024-04-26T21:09:50.224721+0000","last_peered":"2024-04-26T21:09:50.224721+0000","last_clean":"2024-04-26T21:09:50.224721+0000","last_became_active":"2024-04-26T21:09:48.163599+0000","last_became_peered":"2024-04-26T21:09:48.163599+0000","last_unstale":"2024-04-26T21:09:50.224721+0000","last_undegraded":"2024-04-26T21:09:50.224721+0000","last_fullsized":"2024-04-26T21:09:50.224721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:56.909021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.210284+0000","last_change":"2024-04-26T21:09:49.207186+0000","last_active":"2024-04-26T21:09:49.210284+0000","last_peered":"2024-04-26T21:09:49.210284+0000","last_clean":"2024-04-26T21:09:49.210284+0000","last_became_active":"2024-04-26T21:09:49.206907+0000","last_became_peered":"2024-04-26T21:09:49.206907+0000","last_unstale":"2024-04-26T21:09:49.210284+0000","last_undegraded":"2024-04-26T21:09:49.210284+0000","last_fullsized":"2024-04-26T21:09:49.210284+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:18.880151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.247937+0000","last_change":"2024-04-26T21:09:48.169351+0000","last_active":"2024-04-26T21:09:50.247937+0000","last_peered":"2024-04-26T21:09:50.247937+0000","last_clean":"2024-04-26T21:09:50.247937+0000","last_became_active":"2024-04-26T21:09:48.168677+0000","last_became_peered":"2024-04-26T21:09:48.168677+0000","last_unstale":"2024-04-26T21:09:50.247937+0000","last_undegraded":"2024-04-26T21:09:50.247937+0000","last_fullsized":"2024-04-26T21:09:50.247937+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:29.553705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.224180+0000","last_change":"2024-04-26T21:09:49.181995+0000","last_active":"2024-04-26T21:09:50.224180+0000","last_peered":"2024-04-26T21:09:50.224180+0000","last_clean":"2024-04-26T21:09:50.224180+0000","last_became_active":"2024-04-26T21:09:49.181356+0000","last_became_peered":"2024-04-26T21:09:49.181356+0000","last_unstale":"2024-04-26T21:09:50.224180+0000","last_undegraded":"2024-04-26T21:09:50.224180+0000","last_fullsized":"2024-04-26T21:09:50.224180+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:53.758778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.247708+0000","last_change":"2024-04-26T21:09:48.159823+0000","last_active":"2024-04-26T21:09:50.247708+0000","last_peered":"2024-04-26T21:09:50.247708+0000","last_clean":"2024-04-26T21:09:50.247708+0000","last_became_active":"2024-04-26T21:09:48.158773+0000","last_became_peered":"2024-04-26T21:09:48.158773+0000","last_unstale":"2024-04-26T21:09:50.247708+0000","last_undegraded":"2024-04-26T21:09:50.247708+0000","last_fullsized":"2024-04-26T21:09:50.247708+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:11.900171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.204258+0000","last_change":"2024-04-26T21:09:49.193075+0000","last_active":"2024-04-26T21:09:49.204258+0000","last_peered":"2024-04-26T21:09:49.204258+0000","last_clean":"2024-04-26T21:09:49.204258+0000","last_became_active":"2024-04-26T21:09:49.192205+0000","last_became_peered":"2024-04-26T21:09:49.192205+0000","last_unstale":"2024-04-26T21:09:49.204258+0000","last_undegraded":"2024-04-26T21:09:49.204258+0000","last_fullsized":"2024-04-26T21:09:49.204258+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:39:33.860582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.248207+0000","last_change":"2024-04-26T21:09:48.162563+0000","last_active":"2024-04-26T21:09:50.248207+0000","last_peered":"2024-04-26T21:09:50.248207+0000","last_clean":"2024-04-26T21:09:50.248207+0000","last_became_active":"2024-04-26T21:09:48.160696+0000","last_became_peered":"2024-04-26T21:09:48.160696+0000","last_unstale":"2024-04-26T21:09:50.248207+0000","last_undegraded":"2024-04-26T21:09:50.248207+0000","last_fullsized":"2024-04-26T21:09:50.248207+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:11:19.364201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.184912+0000","last_change":"2024-04-26T21:09:49.182396+0000","last_active":"2024-04-26T21:09:49.184912+0000","last_peered":"2024-04-26T21:09:49.184912+0000","last_clean":"2024-04-26T21:09:49.184912+0000","last_became_active":"2024-04-26T21:09:49.181791+0000","last_became_peered":"2024-04-26T21:09:49.181791+0000","last_unstale":"2024-04-26T21:09:49.184912+0000","last_undegraded":"2024-04-26T21:09:49.184912+0000","last_fullsized":"2024-04-26T21:09:49.184912+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:55.773928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.193417+0000","last_change":"2024-04-26T21:09:48.165208+0000","last_active":"2024-04-26T21:09:50.193417+0000","last_peered":"2024-04-26T21:09:50.193417+0000","last_clean":"2024-04-26T21:09:50.193417+0000","last_became_active":"2024-04-26T21:09:48.164876+0000","last_became_peered":"2024-04-26T21:09:48.164876+0000","last_unstale":"2024-04-26T21:09:50.193417+0000","last_undegraded":"2024-04-26T21:09:50.193417+0000","last_fullsized":"2024-04-26T21:09:50.193417+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:07.546830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.204315+0000","last_change":"2024-04-26T21:09:49.194262+0000","last_active":"2024-04-26T21:09:49.204315+0000","last_peered":"2024-04-26T21:09:49.204315+0000","last_clean":"2024-04-26T21:09:49.204315+0000","last_became_active":"2024-04-26T21:09:49.193988+0000","last_became_peered":"2024-04-26T21:09:49.193988+0000","last_unstale":"2024-04-26T21:09:49.204315+0000","last_undegraded":"2024-04-26T21:09:49.204315+0000","last_fullsized":"2024-04-26T21:09:49.204315+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:14:50.294164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.199116+0000","last_change":"2024-04-26T21:09:48.173680+0000","last_active":"2024-04-26T21:09:50.199116+0000","last_peered":"2024-04-26T21:09:50.199116+0000","last_clean":"2024-04-26T21:09:50.199116+0000","last_became_active":"2024-04-26T21:09:48.170159+0000","last_became_peered":"2024-04-26T21:09:48.170159+0000","last_unstale":"2024-04-26T21:09:50.199116+0000","last_undegraded":"2024-04-26T21:09:50.199116+0000","last_fullsized":"2024-04-26T21:09:50.199116+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:39.478970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.204709+0000","last_change":"2024-04-26T21:09:49.196653+0000","last_active":"2024-04-26T21:09:49.204709+0000","last_peered":"2024-04-26T21:09:49.204709+0000","last_clean":"2024-04-26T21:09:49.204709+0000","last_became_active":"2024-04-26T21:09:49.196371+0000","last_became_peered":"2024-04-26T21:09:49.196371+0000","last_unstale":"2024-04-26T21:09:49.204709+0000","last_undegraded":"2024-04-26T21:09:49.204709+0000","last_fullsized":"2024-04-26T21:09:49.204709+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:42.942062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.247097+0000","last_change":"2024-04-26T21:09:48.167011+0000","last_active":"2024-04-26T21:09:50.247097+0000","last_peered":"2024-04-26T21:09:50.247097+0000","last_clean":"2024-04-26T21:09:50.247097+0000","last_became_active":"2024-04-26T21:09:48.164230+0000","last_became_peered":"2024-04-26T21:09:48.164230+0000","last_unstale":"2024-04-26T21:09:50.247097+0000","last_undegraded":"2024-04-26T21:09:50.247097+0000","last_fullsized":"2024-04-26T21:09:50.247097+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:28.162164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.197499+0000","last_change":"2024-04-26T21:09:49.200922+0000","last_active":"2024-04-26T21:09:50.197499+0000","last_peered":"2024-04-26T21:09:50.197499+0000","last_clean":"2024-04-26T21:09:50.197499+0000","last_became_active":"2024-04-26T21:09:49.200611+0000","last_became_peered":"2024-04-26T21:09:49.200611+0000","last_unstale":"2024-04-26T21:09:50.197499+0000","last_undegraded":"2024-04-26T21:09:50.197499+0000","last_fullsized":"2024-04-26T21:09:50.197499+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:37.764086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.189643+0000","last_change":"2024-04-26T21:09:48.175081+0000","last_active":"2024-04-26T21:09:49.189643+0000","last_peered":"2024-04-26T21:09:49.189643+0000","last_clean":"2024-04-26T21:09:49.189643+0000","last_became_active":"2024-04-26T21:09:48.174754+0000","last_became_peered":"2024-04-26T21:09:48.174754+0000","last_unstale":"2024-04-26T21:09:49.189643+0000","last_undegraded":"2024-04-26T21:09:49.189643+0000","last_fullsized":"2024-04-26T21:09:49.189643+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:20.091000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.204821+0000","last_change":"2024-04-26T21:09:49.198727+0000","last_active":"2024-04-26T21:09:49.204821+0000","last_peered":"2024-04-26T21:09:49.204821+0000","last_clean":"2024-04-26T21:09:49.204821+0000","last_became_active":"2024-04-26T21:09:49.198375+0000","last_became_peered":"2024-04-26T21:09:49.198375+0000","last_unstale":"2024-04-26T21:09:49.204821+0000","last_undegraded":"2024-04-26T21:09:49.204821+0000","last_fullsized":"2024-04-26T21:09:49.204821+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:29.177190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.224798+0000","last_change":"2024-04-26T21:09:48.175370+0000","last_active":"2024-04-26T21:09:50.224798+0000","last_peered":"2024-04-26T21:09:50.224798+0000","last_clean":"2024-04-26T21:09:50.224798+0000","last_became_active":"2024-04-26T21:09:48.175068+0000","last_became_peered":"2024-04-26T21:09:48.175068+0000","last_unstale":"2024-04-26T21:09:50.224798+0000","last_undegraded":"2024-04-26T21:09:50.224798+0000","last_fullsized":"2024-04-26T21:09:50.224798+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:07.546830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.212591+0000","last_change":"2024-04-26T21:09:49.209873+0000","last_active":"2024-04-26T21:09:49.212591+0000","last_peered":"2024-04-26T21:09:49.212591+0000","last_clean":"2024-04-26T21:09:49.212591+0000","last_became_active":"2024-04-26T21:09:49.209583+0000","last_became_peered":"2024-04-26T21:09:49.209583+0000","last_unstale":"2024-04-26T21:09:49.212591+0000","last_undegraded":"2024-04-26T21:09:49.212591+0000","last_fullsized":"2024-04-26T21:09:49.212591+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:07:20.080838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.221762+0000","last_change":"2024-04-26T21:09:48.168287+0000","last_active":"2024-04-26T21:09:50.221762+0000","last_peered":"2024-04-26T21:09:50.221762+0000","last_clean":"2024-04-26T21:09:50.221762+0000","last_became_active":"2024-04-26T21:09:48.167956+0000","last_became_peered":"2024-04-26T21:09:48.167956+0000","last_unstale":"2024-04-26T21:09:50.221762+0000","last_undegraded":"2024-04-26T21:09:50.221762+0000","last_fullsized":"2024-04-26T21:09:50.221762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:47:33.049064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.193099+0000","last_change":"2024-04-26T21:09:49.193102+0000","last_active":"2024-04-26T21:09:50.193099+0000","last_peered":"2024-04-26T21:09:50.193099+0000","last_clean":"2024-04-26T21:09:50.193099+0000","last_became_active":"2024-04-26T21:09:49.192772+0000","last_became_peered":"2024-04-26T21:09:49.192772+0000","last_unstale":"2024-04-26T21:09:50.193099+0000","last_undegraded":"2024-04-26T21:09:50.193099+0000","last_fullsized":"2024-04-26T21:09:50.193099+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:19.183103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.224294+0000","last_change":"2024-04-26T21:09:48.162408+0000","last_active":"2024-04-26T21:09:50.224294+0000","last_peered":"2024-04-26T21:09:50.224294+0000","last_clean":"2024-04-26T21:09:50.224294+0000","last_became_active":"2024-04-26T21:09:48.160352+0000","last_became_peered":"2024-04-26T21:09:48.160352+0000","last_unstale":"2024-04-26T21:09:50.224294+0000","last_undegraded":"2024-04-26T21:09:50.224294+0000","last_fullsized":"2024-04-26T21:09:50.224294+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:47:33.049064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.198843+0000","last_change":"2024-04-26T21:09:49.196308+0000","last_active":"2024-04-26T21:09:50.198843+0000","last_peered":"2024-04-26T21:09:50.198843+0000","last_clean":"2024-04-26T21:09:50.198843+0000","last_became_active":"2024-04-26T21:09:49.193660+0000","last_became_peered":"2024-04-26T21:09:49.193660+0000","last_unstale":"2024-04-26T21:09:50.198843+0000","last_undegraded":"2024-04-26T21:09:50.198843+0000","last_fullsized":"2024-04-26T21:09:50.198843+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:36.246542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.193638+0000","last_change":"2024-04-26T21:09:48.178997+0000","last_active":"2024-04-26T21:09:50.193638+0000","last_peered":"2024-04-26T21:09:50.193638+0000","last_clean":"2024-04-26T21:09:50.193638+0000","last_became_active":"2024-04-26T21:09:48.178694+0000","last_became_peered":"2024-04-26T21:09:48.178694+0000","last_unstale":"2024-04-26T21:09:50.193638+0000","last_undegraded":"2024-04-26T21:09:50.193638+0000","last_fullsized":"2024-04-26T21:09:50.193638+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:24:22.909294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.193305+0000","last_change":"2024-04-26T21:09:49.197803+0000","last_active":"2024-04-26T21:09:50.193305+0000","last_peered":"2024-04-26T21:09:50.193305+0000","last_clean":"2024-04-26T21:09:50.193305+0000","last_became_active":"2024-04-26T21:09:49.197547+0000","last_became_peered":"2024-04-26T21:09:49.197547+0000","last_unstale":"2024-04-26T21:09:50.193305+0000","last_undegraded":"2024-04-26T21:09:50.193305+0000","last_fullsized":"2024-04-26T21:09:50.193305+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:07:20.080838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.223801+0000","last_change":"2024-04-26T21:09:48.178326+0000","last_active":"2024-04-26T21:09:50.223801+0000","last_peered":"2024-04-26T21:09:50.223801+0000","last_clean":"2024-04-26T21:09:50.223801+0000","last_became_active":"2024-04-26T21:09:48.178106+0000","last_became_peered":"2024-04-26T21:09:48.178106+0000","last_unstale":"2024-04-26T21:09:50.223801+0000","last_undegraded":"2024-04-26T21:09:50.223801+0000","last_fullsized":"2024-04-26T21:09:50.223801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:04:02.163653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.203152+0000","last_change":"2024-04-26T21:09:49.195241+0000","last_active":"2024-04-26T21:09:49.203152+0000","last_peered":"2024-04-26T21:09:49.203152+0000","last_clean":"2024-04-26T21:09:49.203152+0000","last_became_active":"2024-04-26T21:09:49.194972+0000","last_became_peered":"2024-04-26T21:09:49.194972+0000","last_unstale":"2024-04-26T21:09:49.203152+0000","last_undegraded":"2024-04-26T21:09:49.203152+0000","last_fullsized":"2024-04-26T21:09:49.203152+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:29.134501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.247252+0000","last_change":"2024-04-26T21:09:48.176288+0000","last_active":"2024-04-26T21:09:50.247252+0000","last_peered":"2024-04-26T21:09:50.247252+0000","last_clean":"2024-04-26T21:09:50.247252+0000","last_became_active":"2024-04-26T21:09:48.176044+0000","last_became_peered":"2024-04-26T21:09:48.176044+0000","last_unstale":"2024-04-26T21:09:50.247252+0000","last_undegraded":"2024-04-26T21:09:50.247252+0000","last_fullsized":"2024-04-26T21:09:50.247252+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:08.043747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.246986+0000","last_change":"2024-04-26T21:09:49.194841+0000","last_active":"2024-04-26T21:09:50.246986+0000","last_peered":"2024-04-26T21:09:50.246986+0000","last_clean":"2024-04-26T21:09:50.246986+0000","last_became_active":"2024-04-26T21:09:49.194505+0000","last_became_peered":"2024-04-26T21:09:49.194505+0000","last_unstale":"2024-04-26T21:09:50.246986+0000","last_undegraded":"2024-04-26T21:09:50.246986+0000","last_fullsized":"2024-04-26T21:09:50.246986+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:36.246542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.188004+0000","last_change":"2024-04-26T21:09:48.173297+0000","last_active":"2024-04-26T21:09:49.188004+0000","last_peered":"2024-04-26T21:09:49.188004+0000","last_clean":"2024-04-26T21:09:49.188004+0000","last_became_active":"2024-04-26T21:09:48.172983+0000","last_became_peered":"2024-04-26T21:09:48.172983+0000","last_unstale":"2024-04-26T21:09:49.188004+0000","last_undegraded":"2024-04-26T21:09:49.188004+0000","last_fullsized":"2024-04-26T21:09:49.188004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:07.546830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.197819+0000","last_change":"2024-04-26T21:09:49.210602+0000","last_active":"2024-04-26T21:09:50.197819+0000","last_peered":"2024-04-26T21:09:50.197819+0000","last_clean":"2024-04-26T21:09:50.197819+0000","last_became_active":"2024-04-26T21:09:49.210229+0000","last_became_peered":"2024-04-26T21:09:49.210229+0000","last_unstale":"2024-04-26T21:09:50.197819+0000","last_undegraded":"2024-04-26T21:09:50.197819+0000","last_fullsized":"2024-04-26T21:09:50.197819+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:55:08.786662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.198357+0000","last_change":"2024-04-26T21:09:48.173259+0000","last_active":"2024-04-26T21:09:50.198357+0000","last_peered":"2024-04-26T21:09:50.198357+0000","last_clean":"2024-04-26T21:09:50.198357+0000","last_became_active":"2024-04-26T21:09:48.172960+0000","last_became_peered":"2024-04-26T21:09:48.172960+0000","last_unstale":"2024-04-26T21:09:50.198357+0000","last_undegraded":"2024-04-26T21:09:50.198357+0000","last_fullsized":"2024-04-26T21:09:50.198357+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:29.553705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":193,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.198056+0000","last_change":"2024-04-26T21:08:33.504911+0000","last_active":"2024-04-26T21:09:50.198056+0000","last_peered":"2024-04-26T21:09:50.198056+0000","last_clean":"2024-04-26T21:09:50.198056+0000","last_became_active":"2024-04-26T21:08:33.503667+0000","last_became_peered":"2024-04-26T21:08:33.503667+0000","last_unstale":"2024-04-26T21:09:50.198056+0000","last_undegraded":"2024-04-26T21:09:50.198056+0000","last_fullsized":"2024-04-26T21:09:50.198056+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:32.013907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:32.013907+0000","last_clean_scrub_stamp":"2024-04-26T21:08:32.013907+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:09.596153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":735437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":380928,"data_stored":145595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":145651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":145083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":146613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":376832,"data_stored":145049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":146613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":966656,"data_stored":735347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:09:53.672 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:09:53.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.923+0000 7f82d1e00640 1 Processor -- start 2024-04-26T21:09:53.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.923+0000 7f82d1e00640 1 -- start start 2024-04-26T21:09:53.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.925+0000 7f82d1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f82cc0d1c80 0x7f82cc0d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:53.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.925+0000 7f82d1e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f82cc162ba0 0x7f82cc0d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:53.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.925+0000 7f82d1e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82cc0b61c0 con 0x7f82cc0d2060 2024-04-26T21:09:53.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.925+0000 7f82d1e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f82cc0b5f70 con 0x7f82cc162ba0 2024-04-26T21:09:53.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.925+0000 7f82d1e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f82cc0d3830 con 0x7f82cc0d1c80 2024-04-26T21:09:53.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.925+0000 7f82d1400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f82cc162ba0 0x7f82cc0d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:53.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.925+0000 7f82d1400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f82cc162ba0 0x7f82cc0d0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51092/0 (socket says 172.21.15.26:51092) 2024-04-26T21:09:53.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.925+0000 7f82d1400640 1 -- 172.21.15.26:0/3300557140 learned_addr learned my addr 172.21.15.26:0/3300557140 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:53.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.926+0000 7f82d0a00640 1 --2- 172.21.15.26:0/3300557140 >> v2:172.21.15.86:3300/0 conn(0x7f82cc0d1c80 0x7f82cc0d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:53.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.926+0000 7f82d1400640 1 -- 172.21.15.26:0/3300557140 >> v2:172.21.15.86:3300/0 conn(0x7f82cc0d1c80 msgr2=0x7f82cc0d3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:53.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.926+0000 7f82d1400640 1 --2- 172.21.15.26:0/3300557140 >> v2:172.21.15.86:3300/0 conn(0x7f82cc0d1c80 0x7f82cc0d3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:53.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.926+0000 7f82d1400640 1 -- 172.21.15.26:0/3300557140 >> v1:172.21.15.86:6789/0 conn(0x7f82cc0d2060 legacy=0x7f82cc162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:53.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.926+0000 7f82d1400640 1 -- 172.21.15.26:0/3300557140 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82cc0b64a0 con 0x7f82cc162ba0 2024-04-26T21:09:53.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.927+0000 7f82d1400640 1 --2- 172.21.15.26:0/3300557140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82cc162ba0 0x7f82cc0d0cf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f82cc08d3b0 tx=0x7f82c402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d8fd8cd07a739 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:53.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.927+0000 7f82cb400640 1 -- 172.21.15.26:0/3300557140 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4036070 con 0x7f82cc162ba0 2024-04-26T21:09:53.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.927+0000 7f82cb400640 1 -- 172.21.15.26:0/3300557140 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c4031600 con 0x7f82cc162ba0 2024-04-26T21:09:53.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.927+0000 7f82cb400640 1 -- 172.21.15.26:0/3300557140 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c40319d0 con 0x7f82cc162ba0 2024-04-26T21:09:53.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.928+0000 7f82d1e00640 1 -- 172.21.15.26:0/3300557140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82cc162ba0 msgr2=0x7f82cc0d0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:53.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.928+0000 7f82d1e00640 1 --2- 172.21.15.26:0/3300557140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82cc162ba0 0x7f82cc0d0cf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f82cc08d3b0 tx=0x7f82c402a660 comp rx=0 tx=0).stop 2024-04-26T21:09:53.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.928+0000 7f82d1e00640 1 -- 172.21.15.26:0/3300557140 shutdown_connections 2024-04-26T21:09:53.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.928+0000 7f82d1e00640 1 --2- 172.21.15.26:0/3300557140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82cc162ba0 0x7f82cc0d0cf0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:53.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.928+0000 7f82d1e00640 1 --2- 172.21.15.26:0/3300557140 >> v2:172.21.15.86:3300/0 conn(0x7f82cc0d1c80 0x7f82cc0d3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:53.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.928+0000 7f82d1e00640 1 -- 172.21.15.26:0/3300557140 >> 172.21.15.26:0/3300557140 conn(0x7f82cc0d5020 msgr2=0x7f82cc0d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:53.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.928+0000 7f82d1e00640 1 -- 172.21.15.26:0/3300557140 shutdown_connections 2024-04-26T21:09:53.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.928+0000 7f82d1e00640 1 -- 172.21.15.26:0/3300557140 wait complete. 2024-04-26T21:09:53.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.931+0000 7f82d1e00640 1 Processor -- start 2024-04-26T21:09:53.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.931+0000 7f82d1e00640 1 -- start start 2024-04-26T21:09:53.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.932+0000 7f82d1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82cc0d1c80 0x7f82cc1f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:53.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.932+0000 7f82d1e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82cc0d2060 0x7f82cc0ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:53.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.932+0000 7f82d1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82cc162ba0 0x7f82cc155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:53.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.932+0000 7f82d1e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc0d5cc0 con 0x7f82cc0d1c80 2024-04-26T21:09:53.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.932+0000 7f82d1e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc0d5b40 con 0x7f82cc162ba0 2024-04-26T21:09:53.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.932+0000 7f82d1e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f82cc0c8b70 con 0x7f82cc0d2060 2024-04-26T21:09:53.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.932+0000 7f82d0a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82cc0d1c80 0x7f82cc1f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:53.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.933+0000 7f82d0a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82cc0d1c80 0x7f82cc1f8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51096/0 (socket says 172.21.15.26:51096) 2024-04-26T21:09:53.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.933+0000 7f82d0a00640 1 -- 172.21.15.26:0/3563073644 learned_addr learned my addr 172.21.15.26:0/3563073644 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:53.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.933+0000 7f82cbe00640 1 --2- 172.21.15.26:0/3563073644 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82cc0d2060 0x7f82cc0ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:53.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.933+0000 7f82d1400640 1 --2- 172.21.15.26:0/3563073644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82cc162ba0 0x7f82cc155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:53.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.933+0000 7f82d0a00640 1 -- 172.21.15.26:0/3563073644 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82cc0d2060 msgr2=0x7f82cc0ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:53.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.933+0000 7f82d0a00640 1 --2- 172.21.15.26:0/3563073644 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82cc0d2060 0x7f82cc0ca670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:53.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.933+0000 7f82d0a00640 1 -- 172.21.15.26:0/3563073644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82cc162ba0 msgr2=0x7f82cc155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:53.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.933+0000 7f82d0a00640 1 --2- 172.21.15.26:0/3563073644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82cc162ba0 0x7f82cc155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:53.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.933+0000 7f82d0a00640 1 -- 172.21.15.26:0/3563073644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82cc0cc4d0 con 0x7f82cc0d1c80 2024-04-26T21:09:53.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.934+0000 7f82d0a00640 1 --2- 172.21.15.26:0/3563073644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82cc0d1c80 0x7f82cc1f8f60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f82c000d830 tx=0x7f82c000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:53.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.934+0000 7f82c8c00640 1 -- 172.21.15.26:0/3563073644 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c000b6c0 con 0x7f82cc0d1c80 2024-04-26T21:09:53.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.934+0000 7f82c8c00640 1 -- 172.21.15.26:0/3563073644 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c0019070 con 0x7f82cc0d1c80 2024-04-26T21:09:53.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.934+0000 7f82c8c00640 1 -- 172.21.15.26:0/3563073644 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c000ba80 con 0x7f82cc0d1c80 2024-04-26T21:09:53.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.934+0000 7f82d1e00640 1 -- 172.21.15.26:0/3563073644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82cc0cc7b0 con 0x7f82cc0d1c80 2024-04-26T21:09:53.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.934+0000 7f82d1e00640 1 -- 172.21.15.26:0/3563073644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82cc0ccd70 con 0x7f82cc0d1c80 2024-04-26T21:09:53.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.935+0000 7f82d1e00640 1 -- 172.21.15.26:0/3563073644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82cc0cd150 con 0x7f82cc0d1c80 2024-04-26T21:09:53.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.936+0000 7f82c8c00640 1 -- 172.21.15.26:0/3563073644 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f82c001e460 con 0x7f82cc0d1c80 2024-04-26T21:09:53.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.937+0000 7f82c8c00640 1 --2- 172.21.15.26:0/3563073644 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f82ac06c940 0x7f82ac06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:53.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.937+0000 7f82c8c00640 1 -- 172.21.15.26:0/3563073644 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f82c008c1b0 con 0x7f82cc0d1c80 2024-04-26T21:09:53.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.938+0000 7f82cbe00640 1 --2- 172.21.15.26:0/3563073644 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f82ac06c940 0x7f82ac06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:53.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.938+0000 7f82cbe00640 1 --2- 172.21.15.26:0/3563073644 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f82ac06c940 0x7f82ac06ee00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0cd9d0 tx=0x7f82b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:53.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:53.943+0000 7f82c8c00640 1 -- 172.21.15.26:0/3563073644 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f82cc0cd150 con 0x7f82cc0d1c80 2024-04-26T21:09:54.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.144+0000 7f82d1e00640 1 -- 172.21.15.26:0/3563073644 --> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f82cc06a700 con 0x7f82ac06c940 2024-04-26T21:09:54.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.164+0000 7f82c8c00640 1 -- 172.21.15.26:0/3563073644 <== mgr.4104 v2:172.21.15.26:6832/3130430358 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251920 (secure 0 0 0) 0x7f82cc06a700 con 0x7f82ac06c940 2024-04-26T21:09:54.167 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:54.169 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-26T21:09:54.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.170+0000 7f82d1e00640 1 -- 172.21.15.26:0/3563073644 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f82ac06c940 msgr2=0x7f82ac06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:54.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.170+0000 7f82d1e00640 1 --2- 172.21.15.26:0/3563073644 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f82ac06c940 0x7f82ac06ee00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0cd9d0 tx=0x7f82b8009040 comp rx=0 tx=0).stop 2024-04-26T21:09:54.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.170+0000 7f82d1e00640 1 -- 172.21.15.26:0/3563073644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82cc0d1c80 msgr2=0x7f82cc1f8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:54.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.170+0000 7f82d1e00640 1 --2- 172.21.15.26:0/3563073644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82cc0d1c80 0x7f82cc1f8f60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f82c000d830 tx=0x7f82c000dcf0 comp rx=0 tx=0).stop 2024-04-26T21:09:54.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.170+0000 7f82d1e00640 1 -- 172.21.15.26:0/3563073644 shutdown_connections 2024-04-26T21:09:54.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.170+0000 7f82d1e00640 1 --2- 172.21.15.26:0/3563073644 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f82ac06c940 0x7f82ac06ee00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:54.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.170+0000 7f82d1e00640 1 --2- 172.21.15.26:0/3563073644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82cc162ba0 0x7f82cc155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:54.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.171+0000 7f82d1e00640 1 --2- 172.21.15.26:0/3563073644 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82cc0d2060 0x7f82cc0ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:54.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.171+0000 7f82d1e00640 1 --2- 172.21.15.26:0/3563073644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82cc0d1c80 0x7f82cc1f8f60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:54.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.171+0000 7f82d1e00640 1 -- 172.21.15.26:0/3563073644 >> 172.21.15.26:0/3563073644 conn(0x7f82cc0d5020 msgr2=0x7f82cc15fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:54.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.171+0000 7f82d1e00640 1 -- 172.21.15.26:0/3563073644 shutdown_connections 2024-04-26T21:09:54.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.171+0000 7f82d1e00640 1 -- 172.21.15.26:0/3563073644 wait complete. 2024-04-26T21:09:54.192 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-04-26T21:09:53.311120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236944,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737776,"statfs":{"total":773094113280,"available":772851482624,"internally_reserved":0,"allocated":4235264,"data_stored":2345388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001512"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.198737+0000","last_change":"2024-04-26T21:09:49.209518+0000","last_active":"2024-04-26T21:09:50.198737+0000","last_peered":"2024-04-26T21:09:50.198737+0000","last_clean":"2024-04-26T21:09:50.198737+0000","last_became_active":"2024-04-26T21:09:49.207194+0000","last_became_peered":"2024-04-26T21:09:49.207194+0000","last_unstale":"2024-04-26T21:09:50.198737+0000","last_undegraded":"2024-04-26T21:09:50.198737+0000","last_fullsized":"2024-04-26T21:09:50.198737+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:01.353442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.215396+0000","last_change":"2024-04-26T21:09:49.207992+0000","last_active":"2024-04-26T21:09:49.215396+0000","last_peered":"2024-04-26T21:09:49.215396+0000","last_clean":"2024-04-26T21:09:49.215396+0000","last_became_active":"2024-04-26T21:09:49.207596+0000","last_became_peered":"2024-04-26T21:09:49.207596+0000","last_unstale":"2024-04-26T21:09:49.215396+0000","last_undegraded":"2024-04-26T21:09:49.215396+0000","last_fullsized":"2024-04-26T21:09:49.215396+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:27.951387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.212736+0000","last_change":"2024-04-26T21:09:49.211340+0000","last_active":"2024-04-26T21:09:49.212736+0000","last_peered":"2024-04-26T21:09:49.212736+0000","last_clean":"2024-04-26T21:09:49.212736+0000","last_became_active":"2024-04-26T21:09:49.210123+0000","last_became_peered":"2024-04-26T21:09:49.210123+0000","last_unstale":"2024-04-26T21:09:49.212736+0000","last_undegraded":"2024-04-26T21:09:49.212736+0000","last_fullsized":"2024-04-26T21:09:49.212736+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:27.951387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.223908+0000","last_change":"2024-04-26T21:09:49.191678+0000","last_active":"2024-04-26T21:09:50.223908+0000","last_peered":"2024-04-26T21:09:50.223908+0000","last_clean":"2024-04-26T21:09:50.223908+0000","last_became_active":"2024-04-26T21:09:49.191361+0000","last_became_peered":"2024-04-26T21:09:49.191361+0000","last_unstale":"2024-04-26T21:09:50.223908+0000","last_undegraded":"2024-04-26T21:09:50.223908+0000","last_fullsized":"2024-04-26T21:09:50.223908+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:27.951387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.210297+0000","last_change":"2024-04-26T21:09:49.207043+0000","last_active":"2024-04-26T21:09:49.210297+0000","last_peered":"2024-04-26T21:09:49.210297+0000","last_clean":"2024-04-26T21:09:49.210297+0000","last_became_active":"2024-04-26T21:09:49.206755+0000","last_became_peered":"2024-04-26T21:09:49.206755+0000","last_unstale":"2024-04-26T21:09:49.210297+0000","last_undegraded":"2024-04-26T21:09:49.210297+0000","last_fullsized":"2024-04-26T21:09:49.210297+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:22.050808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.216617+0000","last_change":"2024-04-26T21:09:49.210589+0000","last_active":"2024-04-26T21:09:49.216617+0000","last_peered":"2024-04-26T21:09:49.216617+0000","last_clean":"2024-04-26T21:09:49.216617+0000","last_became_active":"2024-04-26T21:09:49.210194+0000","last_became_peered":"2024-04-26T21:09:49.210194+0000","last_unstale":"2024-04-26T21:09:49.216617+0000","last_undegraded":"2024-04-26T21:09:49.216617+0000","last_fullsized":"2024-04-26T21:09:49.216617+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:37:42.004263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.196845+0000","last_change":"2024-04-26T21:09:49.204070+0000","last_active":"2024-04-26T21:09:50.196845+0000","last_peered":"2024-04-26T21:09:50.196845+0000","last_clean":"2024-04-26T21:09:50.196845+0000","last_became_active":"2024-04-26T21:09:49.202741+0000","last_became_peered":"2024-04-26T21:09:49.202741+0000","last_unstale":"2024-04-26T21:09:50.196845+0000","last_undegraded":"2024-04-26T21:09:50.196845+0000","last_fullsized":"2024-04-26T21:09:50.196845+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:37:42.004263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.196754+0000","last_change":"2024-04-26T21:09:49.205910+0000","last_active":"2024-04-26T21:09:50.196754+0000","last_peered":"2024-04-26T21:09:50.196754+0000","last_clean":"2024-04-26T21:09:50.196754+0000","last_became_active":"2024-04-26T21:09:49.205620+0000","last_became_peered":"2024-04-26T21:09:49.205620+0000","last_unstale":"2024-04-26T21:09:50.196754+0000","last_undegraded":"2024-04-26T21:09:50.196754+0000","last_fullsized":"2024-04-26T21:09:50.196754+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:53.758778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.248215+0000","last_change":"2024-04-26T21:09:49.188874+0000","last_active":"2024-04-26T21:09:50.248215+0000","last_peered":"2024-04-26T21:09:50.248215+0000","last_clean":"2024-04-26T21:09:50.248215+0000","last_became_active":"2024-04-26T21:09:49.187596+0000","last_became_peered":"2024-04-26T21:09:49.187596+0000","last_unstale":"2024-04-26T21:09:50.248215+0000","last_undegraded":"2024-04-26T21:09:50.248215+0000","last_fullsized":"2024-04-26T21:09:50.248215+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:27.951387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.224346+0000","last_change":"2024-04-26T21:09:49.211984+0000","last_active":"2024-04-26T21:09:50.224346+0000","last_peered":"2024-04-26T21:09:50.224346+0000","last_clean":"2024-04-26T21:09:50.224346+0000","last_became_active":"2024-04-26T21:09:49.211637+0000","last_became_peered":"2024-04-26T21:09:49.211637+0000","last_unstale":"2024-04-26T21:09:50.224346+0000","last_undegraded":"2024-04-26T21:09:50.224346+0000","last_fullsized":"2024-04-26T21:09:50.224346+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:35.597618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.197962+0000","last_change":"2024-04-26T21:09:49.217009+0000","last_active":"2024-04-26T21:09:50.197962+0000","last_peered":"2024-04-26T21:09:50.197962+0000","last_clean":"2024-04-26T21:09:50.197962+0000","last_became_active":"2024-04-26T21:09:49.216635+0000","last_became_peered":"2024-04-26T21:09:49.216635+0000","last_unstale":"2024-04-26T21:09:50.197962+0000","last_undegraded":"2024-04-26T21:09:50.197962+0000","last_fullsized":"2024-04-26T21:09:50.197962+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:10:58.695182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.207580+0000","last_change":"2024-04-26T21:09:49.202752+0000","last_active":"2024-04-26T21:09:49.207580+0000","last_peered":"2024-04-26T21:09:49.207580+0000","last_clean":"2024-04-26T21:09:49.207580+0000","last_became_active":"2024-04-26T21:09:49.202103+0000","last_became_peered":"2024-04-26T21:09:49.202103+0000","last_unstale":"2024-04-26T21:09:49.207580+0000","last_undegraded":"2024-04-26T21:09:49.207580+0000","last_fullsized":"2024-04-26T21:09:49.207580+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:36.246542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.188106+0000","last_change":"2024-04-26T21:09:49.186462+0000","last_active":"2024-04-26T21:09:49.188106+0000","last_peered":"2024-04-26T21:09:49.188106+0000","last_clean":"2024-04-26T21:09:49.188106+0000","last_became_active":"2024-04-26T21:09:49.185938+0000","last_became_peered":"2024-04-26T21:09:49.185938+0000","last_unstale":"2024-04-26T21:09:49.188106+0000","last_undegraded":"2024-04-26T21:09:49.188106+0000","last_fullsized":"2024-04-26T21:09:49.188106+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:45.675322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.212215+0000","last_change":"2024-04-26T21:09:49.203532+0000","last_active":"2024-04-26T21:09:49.212215+0000","last_peered":"2024-04-26T21:09:49.212215+0000","last_clean":"2024-04-26T21:09:49.212215+0000","last_became_active":"2024-04-26T21:09:49.203232+0000","last_became_peered":"2024-04-26T21:09:49.203232+0000","last_unstale":"2024-04-26T21:09:49.212215+0000","last_undegraded":"2024-04-26T21:09:49.212215+0000","last_fullsized":"2024-04-26T21:09:49.212215+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:18.403639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.207531+0000","last_change":"2024-04-26T21:09:49.203170+0000","last_active":"2024-04-26T21:09:49.207531+0000","last_peered":"2024-04-26T21:09:49.207531+0000","last_clean":"2024-04-26T21:09:49.207531+0000","last_became_active":"2024-04-26T21:09:49.202878+0000","last_became_peered":"2024-04-26T21:09:49.202878+0000","last_unstale":"2024-04-26T21:09:49.207531+0000","last_undegraded":"2024-04-26T21:09:49.207531+0000","last_fullsized":"2024-04-26T21:09:49.207531+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:29.134501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.197962+0000","last_change":"2024-04-26T21:09:49.217585+0000","last_active":"2024-04-26T21:09:50.197962+0000","last_peered":"2024-04-26T21:09:50.197962+0000","last_clean":"2024-04-26T21:09:50.197962+0000","last_became_active":"2024-04-26T21:09:49.217286+0000","last_became_peered":"2024-04-26T21:09:49.217286+0000","last_unstale":"2024-04-26T21:09:50.197962+0000","last_undegraded":"2024-04-26T21:09:50.197962+0000","last_fullsized":"2024-04-26T21:09:50.197962+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:20.043920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.198987+0000","last_change":"2024-04-26T21:09:49.207303+0000","last_active":"2024-04-26T21:09:50.198987+0000","last_peered":"2024-04-26T21:09:50.198987+0000","last_clean":"2024-04-26T21:09:50.198987+0000","last_became_active":"2024-04-26T21:09:49.207016+0000","last_became_peered":"2024-04-26T21:09:49.207016+0000","last_unstale":"2024-04-26T21:09:50.198987+0000","last_undegraded":"2024-04-26T21:09:50.198987+0000","last_fullsized":"2024-04-26T21:09:50.198987+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:36:27.020554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.218189+0000","last_change":"2024-04-26T21:09:49.215174+0000","last_active":"2024-04-26T21:09:49.218189+0000","last_peered":"2024-04-26T21:09:49.218189+0000","last_clean":"2024-04-26T21:09:49.218189+0000","last_became_active":"2024-04-26T21:09:49.213199+0000","last_became_peered":"2024-04-26T21:09:49.213199+0000","last_unstale":"2024-04-26T21:09:49.218189+0000","last_undegraded":"2024-04-26T21:09:49.218189+0000","last_fullsized":"2024-04-26T21:09:49.218189+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:51.306403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.216737+0000","last_change":"2024-04-26T21:09:49.212245+0000","last_active":"2024-04-26T21:09:49.216737+0000","last_peered":"2024-04-26T21:09:49.216737+0000","last_clean":"2024-04-26T21:09:49.216737+0000","last_became_active":"2024-04-26T21:09:49.211925+0000","last_became_peered":"2024-04-26T21:09:49.211925+0000","last_unstale":"2024-04-26T21:09:49.216737+0000","last_undegraded":"2024-04-26T21:09:49.216737+0000","last_fullsized":"2024-04-26T21:09:49.216737+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:35.597618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.208340+0000","last_change":"2024-04-26T21:09:49.204645+0000","last_active":"2024-04-26T21:09:49.208340+0000","last_peered":"2024-04-26T21:09:49.208340+0000","last_clean":"2024-04-26T21:09:49.208340+0000","last_became_active":"2024-04-26T21:09:49.204326+0000","last_became_peered":"2024-04-26T21:09:49.204326+0000","last_unstale":"2024-04-26T21:09:49.208340+0000","last_undegraded":"2024-04-26T21:09:49.208340+0000","last_fullsized":"2024-04-26T21:09:49.208340+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:09.058773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.247656+0000","last_change":"2024-04-26T21:09:49.194892+0000","last_active":"2024-04-26T21:09:50.247656+0000","last_peered":"2024-04-26T21:09:50.247656+0000","last_clean":"2024-04-26T21:09:50.247656+0000","last_became_active":"2024-04-26T21:09:49.194359+0000","last_became_peered":"2024-04-26T21:09:49.194359+0000","last_unstale":"2024-04-26T21:09:50.247656+0000","last_undegraded":"2024-04-26T21:09:50.247656+0000","last_fullsized":"2024-04-26T21:09:50.247656+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:18.880151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.247663+0000","last_change":"2024-04-26T21:09:49.201279+0000","last_active":"2024-04-26T21:09:50.247663+0000","last_peered":"2024-04-26T21:09:50.247663+0000","last_clean":"2024-04-26T21:09:50.247663+0000","last_became_active":"2024-04-26T21:09:49.200954+0000","last_became_peered":"2024-04-26T21:09:49.200954+0000","last_unstale":"2024-04-26T21:09:50.247663+0000","last_undegraded":"2024-04-26T21:09:50.247663+0000","last_fullsized":"2024-04-26T21:09:50.247663+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:07:20.080838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.192766+0000","last_change":"2024-04-26T21:09:49.194084+0000","last_active":"2024-04-26T21:09:50.192766+0000","last_peered":"2024-04-26T21:09:50.192766+0000","last_clean":"2024-04-26T21:09:50.192766+0000","last_became_active":"2024-04-26T21:09:49.193781+0000","last_became_peered":"2024-04-26T21:09:49.193781+0000","last_unstale":"2024-04-26T21:09:50.192766+0000","last_undegraded":"2024-04-26T21:09:50.192766+0000","last_fullsized":"2024-04-26T21:09:50.192766+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:29.134501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.209144+0000","last_change":"2024-04-26T21:09:49.205033+0000","last_active":"2024-04-26T21:09:49.209144+0000","last_peered":"2024-04-26T21:09:49.209144+0000","last_clean":"2024-04-26T21:09:49.209144+0000","last_became_active":"2024-04-26T21:09:49.204717+0000","last_became_peered":"2024-04-26T21:09:49.204717+0000","last_unstale":"2024-04-26T21:09:49.209144+0000","last_undegraded":"2024-04-26T21:09:49.209144+0000","last_fullsized":"2024-04-26T21:09:49.209144+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:19.183103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.224599+0000","last_change":"2024-04-26T21:09:49.209351+0000","last_active":"2024-04-26T21:09:50.224599+0000","last_peered":"2024-04-26T21:09:50.224599+0000","last_clean":"2024-04-26T21:09:50.224599+0000","last_became_active":"2024-04-26T21:09:49.209042+0000","last_became_peered":"2024-04-26T21:09:49.209042+0000","last_unstale":"2024-04-26T21:09:50.224599+0000","last_undegraded":"2024-04-26T21:09:50.224599+0000","last_fullsized":"2024-04-26T21:09:50.224599+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:27.737169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.215375+0000","last_change":"2024-04-26T21:09:49.203091+0000","last_active":"2024-04-26T21:09:49.215375+0000","last_peered":"2024-04-26T21:09:49.215375+0000","last_clean":"2024-04-26T21:09:49.215375+0000","last_became_active":"2024-04-26T21:09:49.200996+0000","last_became_peered":"2024-04-26T21:09:49.200996+0000","last_unstale":"2024-04-26T21:09:49.215375+0000","last_undegraded":"2024-04-26T21:09:49.215375+0000","last_fullsized":"2024-04-26T21:09:49.215375+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:07:20.080838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.224383+0000","last_change":"2024-04-26T21:09:49.214882+0000","last_active":"2024-04-26T21:09:50.224383+0000","last_peered":"2024-04-26T21:09:50.224383+0000","last_clean":"2024-04-26T21:09:50.224383+0000","last_became_active":"2024-04-26T21:09:49.214023+0000","last_became_peered":"2024-04-26T21:09:49.214023+0000","last_unstale":"2024-04-26T21:09:50.224383+0000","last_undegraded":"2024-04-26T21:09:50.224383+0000","last_fullsized":"2024-04-26T21:09:50.224383+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:51.306403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.206386+0000","last_change":"2024-04-26T21:09:49.201229+0000","last_active":"2024-04-26T21:09:49.206386+0000","last_peered":"2024-04-26T21:09:49.206386+0000","last_clean":"2024-04-26T21:09:49.206386+0000","last_became_active":"2024-04-26T21:09:49.200412+0000","last_became_peered":"2024-04-26T21:09:49.200412+0000","last_unstale":"2024-04-26T21:09:49.206386+0000","last_undegraded":"2024-04-26T21:09:49.206386+0000","last_fullsized":"2024-04-26T21:09:49.206386+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:01.353442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.248182+0000","last_change":"2024-04-26T21:09:49.195095+0000","last_active":"2024-04-26T21:09:50.248182+0000","last_peered":"2024-04-26T21:09:50.248182+0000","last_clean":"2024-04-26T21:09:50.248182+0000","last_became_active":"2024-04-26T21:09:49.194783+0000","last_became_peered":"2024-04-26T21:09:49.194783+0000","last_unstale":"2024-04-26T21:09:50.248182+0000","last_undegraded":"2024-04-26T21:09:50.248182+0000","last_fullsized":"2024-04-26T21:09:50.248182+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:18.403639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.196904+0000","last_change":"2024-04-26T21:09:49.205089+0000","last_active":"2024-04-26T21:09:50.196904+0000","last_peered":"2024-04-26T21:09:50.196904+0000","last_clean":"2024-04-26T21:09:50.196904+0000","last_became_active":"2024-04-26T21:09:49.204713+0000","last_became_peered":"2024-04-26T21:09:49.204713+0000","last_unstale":"2024-04-26T21:09:50.196904+0000","last_undegraded":"2024-04-26T21:09:50.196904+0000","last_fullsized":"2024-04-26T21:09:50.196904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:35.597618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.197603+0000","last_change":"2024-04-26T21:09:49.196401+0000","last_active":"2024-04-26T21:09:50.197603+0000","last_peered":"2024-04-26T21:09:50.197603+0000","last_clean":"2024-04-26T21:09:50.197603+0000","last_became_active":"2024-04-26T21:09:49.193990+0000","last_became_peered":"2024-04-26T21:09:49.193990+0000","last_unstale":"2024-04-26T21:09:50.197603+0000","last_undegraded":"2024-04-26T21:09:50.197603+0000","last_fullsized":"2024-04-26T21:09:50.197603+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:17.572409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.197147+0000","last_change":"2024-04-26T21:09:49.199637+0000","last_active":"2024-04-26T21:09:50.197147+0000","last_peered":"2024-04-26T21:09:50.197147+0000","last_clean":"2024-04-26T21:09:50.197147+0000","last_became_active":"2024-04-26T21:09:49.199067+0000","last_became_peered":"2024-04-26T21:09:49.199067+0000","last_unstale":"2024-04-26T21:09:50.197147+0000","last_undegraded":"2024-04-26T21:09:50.197147+0000","last_fullsized":"2024-04-26T21:09:50.197147+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:27.737169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.193227+0000","last_change":"2024-04-26T21:09:49.196346+0000","last_active":"2024-04-26T21:09:50.193227+0000","last_peered":"2024-04-26T21:09:50.193227+0000","last_clean":"2024-04-26T21:09:50.193227+0000","last_became_active":"2024-04-26T21:09:49.196123+0000","last_became_peered":"2024-04-26T21:09:49.196123+0000","last_unstale":"2024-04-26T21:09:50.193227+0000","last_undegraded":"2024-04-26T21:09:50.193227+0000","last_fullsized":"2024-04-26T21:09:50.193227+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:18.880151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.199198+0000","last_change":"2024-04-26T21:09:48.178859+0000","last_active":"2024-04-26T21:09:50.199198+0000","last_peered":"2024-04-26T21:09:50.199198+0000","last_clean":"2024-04-26T21:09:50.199198+0000","last_became_active":"2024-04-26T21:09:48.178328+0000","last_became_peered":"2024-04-26T21:09:48.178328+0000","last_unstale":"2024-04-26T21:09:50.199198+0000","last_undegraded":"2024-04-26T21:09:50.199198+0000","last_fullsized":"2024-04-26T21:09:50.199198+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:11.900171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.198926+0000","last_change":"2024-04-26T21:09:49.199164+0000","last_active":"2024-04-26T21:09:50.198926+0000","last_peered":"2024-04-26T21:09:50.198926+0000","last_clean":"2024-04-26T21:09:50.198926+0000","last_became_active":"2024-04-26T21:09:49.198852+0000","last_became_peered":"2024-04-26T21:09:49.198852+0000","last_unstale":"2024-04-26T21:09:50.198926+0000","last_undegraded":"2024-04-26T21:09:50.198926+0000","last_fullsized":"2024-04-26T21:09:50.198926+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:29.134501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.193186+0000","last_change":"2024-04-26T21:09:48.174461+0000","last_active":"2024-04-26T21:09:50.193186+0000","last_peered":"2024-04-26T21:09:50.193186+0000","last_clean":"2024-04-26T21:09:50.193186+0000","last_became_active":"2024-04-26T21:09:48.173832+0000","last_became_peered":"2024-04-26T21:09:48.173832+0000","last_unstale":"2024-04-26T21:09:50.193186+0000","last_undegraded":"2024-04-26T21:09:50.193186+0000","last_fullsized":"2024-04-26T21:09:50.193186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:04:02.163653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.248364+0000","last_change":"2024-04-26T21:09:49.201270+0000","last_active":"2024-04-26T21:09:50.248364+0000","last_peered":"2024-04-26T21:09:50.248364+0000","last_clean":"2024-04-26T21:09:50.248364+0000","last_became_active":"2024-04-26T21:09:49.200958+0000","last_became_peered":"2024-04-26T21:09:49.200958+0000","last_unstale":"2024-04-26T21:09:50.248364+0000","last_undegraded":"2024-04-26T21:09:50.248364+0000","last_fullsized":"2024-04-26T21:09:50.248364+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:53.758778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.230685+0000","last_change":"2024-04-26T21:09:48.162213+0000","last_active":"2024-04-26T21:09:49.230685+0000","last_peered":"2024-04-26T21:09:49.230685+0000","last_clean":"2024-04-26T21:09:49.230685+0000","last_became_active":"2024-04-26T21:09:48.161912+0000","last_became_peered":"2024-04-26T21:09:48.161912+0000","last_unstale":"2024-04-26T21:09:49.230685+0000","last_undegraded":"2024-04-26T21:09:49.230685+0000","last_fullsized":"2024-04-26T21:09:49.230685+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:20.091000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.224295+0000","last_change":"2024-04-26T21:09:49.198708+0000","last_active":"2024-04-26T21:09:50.224295+0000","last_peered":"2024-04-26T21:09:50.224295+0000","last_clean":"2024-04-26T21:09:50.224295+0000","last_became_active":"2024-04-26T21:09:49.192637+0000","last_became_peered":"2024-04-26T21:09:49.192637+0000","last_unstale":"2024-04-26T21:09:50.224295+0000","last_undegraded":"2024-04-26T21:09:50.224295+0000","last_fullsized":"2024-04-26T21:09:50.224295+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:37:42.004263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.226886+0000","last_change":"2024-04-26T21:09:48.166374+0000","last_active":"2024-04-26T21:09:49.226886+0000","last_peered":"2024-04-26T21:09:49.226886+0000","last_clean":"2024-04-26T21:09:49.226886+0000","last_became_active":"2024-04-26T21:09:48.166078+0000","last_became_peered":"2024-04-26T21:09:48.166078+0000","last_unstale":"2024-04-26T21:09:49.226886+0000","last_undegraded":"2024-04-26T21:09:49.226886+0000","last_fullsized":"2024-04-26T21:09:49.226886+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:31:10.713295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.197009+0000","last_change":"2024-04-26T21:09:49.210181+0000","last_active":"2024-04-26T21:09:50.197009+0000","last_peered":"2024-04-26T21:09:50.197009+0000","last_clean":"2024-04-26T21:09:50.197009+0000","last_became_active":"2024-04-26T21:09:49.209628+0000","last_became_peered":"2024-04-26T21:09:49.209628+0000","last_unstale":"2024-04-26T21:09:50.197009+0000","last_undegraded":"2024-04-26T21:09:50.197009+0000","last_fullsized":"2024-04-26T21:09:50.197009+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:51.306403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.248013+0000","last_change":"2024-04-26T21:09:48.157178+0000","last_active":"2024-04-26T21:09:50.248013+0000","last_peered":"2024-04-26T21:09:50.248013+0000","last_clean":"2024-04-26T21:09:50.248013+0000","last_became_active":"2024-04-26T21:09:48.156051+0000","last_became_peered":"2024-04-26T21:09:48.156051+0000","last_unstale":"2024-04-26T21:09:50.248013+0000","last_undegraded":"2024-04-26T21:09:50.248013+0000","last_fullsized":"2024-04-26T21:09:50.248013+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:51:05.346543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.206474+0000","last_change":"2024-04-26T21:09:49.198957+0000","last_active":"2024-04-26T21:09:49.206474+0000","last_peered":"2024-04-26T21:09:49.206474+0000","last_clean":"2024-04-26T21:09:49.206474+0000","last_became_active":"2024-04-26T21:09:49.198644+0000","last_became_peered":"2024-04-26T21:09:49.198644+0000","last_unstale":"2024-04-26T21:09:49.206474+0000","last_undegraded":"2024-04-26T21:09:49.206474+0000","last_fullsized":"2024-04-26T21:09:49.206474+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:31.125270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.247354+0000","last_change":"2024-04-26T21:09:48.161763+0000","last_active":"2024-04-26T21:09:50.247354+0000","last_peered":"2024-04-26T21:09:50.247354+0000","last_clean":"2024-04-26T21:09:50.247354+0000","last_became_active":"2024-04-26T21:09:48.161279+0000","last_became_peered":"2024-04-26T21:09:48.161279+0000","last_unstale":"2024-04-26T21:09:50.247354+0000","last_undegraded":"2024-04-26T21:09:50.247354+0000","last_fullsized":"2024-04-26T21:09:50.247354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:21.035783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.247279+0000","last_change":"2024-04-26T21:09:49.196379+0000","last_active":"2024-04-26T21:09:50.247279+0000","last_peered":"2024-04-26T21:09:50.247279+0000","last_clean":"2024-04-26T21:09:50.247279+0000","last_became_active":"2024-04-26T21:09:49.196087+0000","last_became_peered":"2024-04-26T21:09:49.196087+0000","last_unstale":"2024-04-26T21:09:50.247279+0000","last_undegraded":"2024-04-26T21:09:50.247279+0000","last_fullsized":"2024-04-26T21:09:50.247279+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:29.134501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.188775+0000","last_change":"2024-04-26T21:09:48.145300+0000","last_active":"2024-04-26T21:09:49.188775+0000","last_peered":"2024-04-26T21:09:49.188775+0000","last_clean":"2024-04-26T21:09:49.188775+0000","last_became_active":"2024-04-26T21:09:48.144878+0000","last_became_peered":"2024-04-26T21:09:48.144878+0000","last_unstale":"2024-04-26T21:09:49.188775+0000","last_undegraded":"2024-04-26T21:09:49.188775+0000","last_fullsized":"2024-04-26T21:09:49.188775+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:30.110244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.223630+0000","last_change":"2024-04-26T21:09:49.209773+0000","last_active":"2024-04-26T21:09:50.223630+0000","last_peered":"2024-04-26T21:09:50.223630+0000","last_clean":"2024-04-26T21:09:50.223630+0000","last_became_active":"2024-04-26T21:09:49.209401+0000","last_became_peered":"2024-04-26T21:09:49.209401+0000","last_unstale":"2024-04-26T21:09:50.223630+0000","last_undegraded":"2024-04-26T21:09:50.223630+0000","last_fullsized":"2024-04-26T21:09:50.223630+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:07:20.080838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.247080+0000","last_change":"2024-04-26T21:09:48.165782+0000","last_active":"2024-04-26T21:09:50.247080+0000","last_peered":"2024-04-26T21:09:50.247080+0000","last_clean":"2024-04-26T21:09:50.247080+0000","last_became_active":"2024-04-26T21:09:48.165428+0000","last_became_peered":"2024-04-26T21:09:48.165428+0000","last_unstale":"2024-04-26T21:09:50.247080+0000","last_undegraded":"2024-04-26T21:09:50.247080+0000","last_fullsized":"2024-04-26T21:09:50.247080+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:41.927036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.218163+0000","last_change":"2024-04-26T21:09:49.216343+0000","last_active":"2024-04-26T21:09:49.218163+0000","last_peered":"2024-04-26T21:09:49.218163+0000","last_clean":"2024-04-26T21:09:49.218163+0000","last_became_active":"2024-04-26T21:09:49.216055+0000","last_became_peered":"2024-04-26T21:09:49.216055+0000","last_unstale":"2024-04-26T21:09:49.218163+0000","last_undegraded":"2024-04-26T21:09:49.218163+0000","last_fullsized":"2024-04-26T21:09:49.218163+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:53.758778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.189024+0000","last_change":"2024-04-26T21:09:48.173351+0000","last_active":"2024-04-26T21:09:49.189024+0000","last_peered":"2024-04-26T21:09:49.189024+0000","last_clean":"2024-04-26T21:09:49.189024+0000","last_became_active":"2024-04-26T21:09:48.173109+0000","last_became_peered":"2024-04-26T21:09:48.173109+0000","last_unstale":"2024-04-26T21:09:49.189024+0000","last_undegraded":"2024-04-26T21:09:49.189024+0000","last_fullsized":"2024-04-26T21:09:49.189024+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:35.231516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.212999+0000","last_change":"2024-04-26T21:09:49.208784+0000","last_active":"2024-04-26T21:09:49.212999+0000","last_peered":"2024-04-26T21:09:49.212999+0000","last_clean":"2024-04-26T21:09:49.212999+0000","last_became_active":"2024-04-26T21:09:49.208510+0000","last_became_peered":"2024-04-26T21:09:49.208510+0000","last_unstale":"2024-04-26T21:09:49.212999+0000","last_undegraded":"2024-04-26T21:09:49.212999+0000","last_fullsized":"2024-04-26T21:09:49.212999+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:53.758778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.193312+0000","last_change":"2024-04-26T21:09:48.159974+0000","last_active":"2024-04-26T21:09:50.193312+0000","last_peered":"2024-04-26T21:09:50.193312+0000","last_clean":"2024-04-26T21:09:50.193312+0000","last_became_active":"2024-04-26T21:09:48.159623+0000","last_became_peered":"2024-04-26T21:09:48.159623+0000","last_unstale":"2024-04-26T21:09:50.193312+0000","last_undegraded":"2024-04-26T21:09:50.193312+0000","last_fullsized":"2024-04-26T21:09:50.193312+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:56.909021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.205877+0000","last_change":"2024-04-26T21:09:49.199148+0000","last_active":"2024-04-26T21:09:49.205877+0000","last_peered":"2024-04-26T21:09:49.205877+0000","last_clean":"2024-04-26T21:09:49.205877+0000","last_became_active":"2024-04-26T21:09:49.198829+0000","last_became_peered":"2024-04-26T21:09:49.198829+0000","last_unstale":"2024-04-26T21:09:49.205877+0000","last_undegraded":"2024-04-26T21:09:49.205877+0000","last_fullsized":"2024-04-26T21:09:49.205877+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:19.183103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.229518+0000","last_change":"2024-04-26T21:09:48.178060+0000","last_active":"2024-04-26T21:09:49.229518+0000","last_peered":"2024-04-26T21:09:49.229518+0000","last_clean":"2024-04-26T21:09:49.229518+0000","last_became_active":"2024-04-26T21:09:48.177786+0000","last_became_peered":"2024-04-26T21:09:48.177786+0000","last_unstale":"2024-04-26T21:09:49.229518+0000","last_undegraded":"2024-04-26T21:09:49.229518+0000","last_fullsized":"2024-04-26T21:09:49.229518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:16.557384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.212713+0000","last_change":"2024-04-26T21:09:49.211074+0000","last_active":"2024-04-26T21:09:49.212713+0000","last_peered":"2024-04-26T21:09:49.212713+0000","last_clean":"2024-04-26T21:09:49.212713+0000","last_became_active":"2024-04-26T21:09:49.210724+0000","last_became_peered":"2024-04-26T21:09:49.210724+0000","last_unstale":"2024-04-26T21:09:49.212713+0000","last_undegraded":"2024-04-26T21:09:49.212713+0000","last_fullsized":"2024-04-26T21:09:49.212713+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:18.403639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.234430+0000","last_change":"2024-04-26T21:09:48.165226+0000","last_active":"2024-04-26T21:09:49.234430+0000","last_peered":"2024-04-26T21:09:49.234430+0000","last_clean":"2024-04-26T21:09:49.234430+0000","last_became_active":"2024-04-26T21:09:48.164925+0000","last_became_peered":"2024-04-26T21:09:48.164925+0000","last_unstale":"2024-04-26T21:09:49.234430+0000","last_undegraded":"2024-04-26T21:09:49.234430+0000","last_fullsized":"2024-04-26T21:09:49.234430+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:00.338416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.223703+0000","last_change":"2024-04-26T21:09:49.209883+0000","last_active":"2024-04-26T21:09:50.223703+0000","last_peered":"2024-04-26T21:09:50.223703+0000","last_clean":"2024-04-26T21:09:50.223703+0000","last_became_active":"2024-04-26T21:09:49.209072+0000","last_became_peered":"2024-04-26T21:09:49.209072+0000","last_unstale":"2024-04-26T21:09:50.223703+0000","last_undegraded":"2024-04-26T21:09:50.223703+0000","last_fullsized":"2024-04-26T21:09:50.223703+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:18.403639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.248213+0000","last_change":"2024-04-26T21:09:48.170697+0000","last_active":"2024-04-26T21:09:50.248213+0000","last_peered":"2024-04-26T21:09:50.248213+0000","last_clean":"2024-04-26T21:09:50.248213+0000","last_became_active":"2024-04-26T21:09:48.169662+0000","last_became_peered":"2024-04-26T21:09:48.169662+0000","last_unstale":"2024-04-26T21:09:50.248213+0000","last_undegraded":"2024-04-26T21:09:50.248213+0000","last_fullsized":"2024-04-26T21:09:50.248213+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:39.478970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.207536+0000","last_change":"2024-04-26T21:09:49.200924+0000","last_active":"2024-04-26T21:09:49.207536+0000","last_peered":"2024-04-26T21:09:49.207536+0000","last_clean":"2024-04-26T21:09:49.207536+0000","last_became_active":"2024-04-26T21:09:49.200546+0000","last_became_peered":"2024-04-26T21:09:49.200546+0000","last_unstale":"2024-04-26T21:09:49.207536+0000","last_undegraded":"2024-04-26T21:09:49.207536+0000","last_fullsized":"2024-04-26T21:09:49.207536+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:18.880151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.223798+0000","last_change":"2024-04-26T21:09:48.169726+0000","last_active":"2024-04-26T21:09:50.223798+0000","last_peered":"2024-04-26T21:09:50.223798+0000","last_clean":"2024-04-26T21:09:50.223798+0000","last_became_active":"2024-04-26T21:09:48.169328+0000","last_became_peered":"2024-04-26T21:09:48.169328+0000","last_unstale":"2024-04-26T21:09:50.223798+0000","last_undegraded":"2024-04-26T21:09:50.223798+0000","last_fullsized":"2024-04-26T21:09:50.223798+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:11.900171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.197582+0000","last_change":"2024-04-26T21:09:49.199729+0000","last_active":"2024-04-26T21:09:50.197582+0000","last_peered":"2024-04-26T21:09:50.197582+0000","last_clean":"2024-04-26T21:09:50.197582+0000","last_became_active":"2024-04-26T21:09:49.199395+0000","last_became_peered":"2024-04-26T21:09:49.199395+0000","last_unstale":"2024-04-26T21:09:50.197582+0000","last_undegraded":"2024-04-26T21:09:50.197582+0000","last_fullsized":"2024-04-26T21:09:50.197582+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:46.120808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.188442+0000","last_change":"2024-04-26T21:09:48.172714+0000","last_active":"2024-04-26T21:09:49.188442+0000","last_peered":"2024-04-26T21:09:49.188442+0000","last_clean":"2024-04-26T21:09:49.188442+0000","last_became_active":"2024-04-26T21:09:48.172407+0000","last_became_peered":"2024-04-26T21:09:48.172407+0000","last_unstale":"2024-04-26T21:09:49.188442+0000","last_undegraded":"2024-04-26T21:09:49.188442+0000","last_fullsized":"2024-04-26T21:09:49.188442+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:50:16.557384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.197677+0000","last_change":"2024-04-26T21:09:49.197339+0000","last_active":"2024-04-26T21:09:50.197677+0000","last_peered":"2024-04-26T21:09:50.197677+0000","last_clean":"2024-04-26T21:09:50.197677+0000","last_became_active":"2024-04-26T21:09:49.197051+0000","last_became_peered":"2024-04-26T21:09:49.197051+0000","last_unstale":"2024-04-26T21:09:50.197677+0000","last_undegraded":"2024-04-26T21:09:50.197677+0000","last_fullsized":"2024-04-26T21:09:50.197677+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:31.125270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.234005+0000","last_change":"2024-04-26T21:09:48.165303+0000","last_active":"2024-04-26T21:09:49.234005+0000","last_peered":"2024-04-26T21:09:49.234005+0000","last_clean":"2024-04-26T21:09:49.234005+0000","last_became_active":"2024-04-26T21:09:48.164809+0000","last_became_peered":"2024-04-26T21:09:48.164809+0000","last_unstale":"2024-04-26T21:09:49.234005+0000","last_undegraded":"2024-04-26T21:09:49.234005+0000","last_fullsized":"2024-04-26T21:09:49.234005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:24:22.909294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.247601+0000","last_change":"2024-04-26T21:09:49.204572+0000","last_active":"2024-04-26T21:09:50.247601+0000","last_peered":"2024-04-26T21:09:50.247601+0000","last_clean":"2024-04-26T21:09:50.247601+0000","last_became_active":"2024-04-26T21:09:49.204183+0000","last_became_peered":"2024-04-26T21:09:49.204183+0000","last_unstale":"2024-04-26T21:09:50.247601+0000","last_undegraded":"2024-04-26T21:09:50.247601+0000","last_fullsized":"2024-04-26T21:09:50.247601+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:19.183103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.224721+0000","last_change":"2024-04-26T21:09:48.164054+0000","last_active":"2024-04-26T21:09:50.224721+0000","last_peered":"2024-04-26T21:09:50.224721+0000","last_clean":"2024-04-26T21:09:50.224721+0000","last_became_active":"2024-04-26T21:09:48.163599+0000","last_became_peered":"2024-04-26T21:09:48.163599+0000","last_unstale":"2024-04-26T21:09:50.224721+0000","last_undegraded":"2024-04-26T21:09:50.224721+0000","last_fullsized":"2024-04-26T21:09:50.224721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:33:56.909021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.210284+0000","last_change":"2024-04-26T21:09:49.207186+0000","last_active":"2024-04-26T21:09:49.210284+0000","last_peered":"2024-04-26T21:09:49.210284+0000","last_clean":"2024-04-26T21:09:49.210284+0000","last_became_active":"2024-04-26T21:09:49.206907+0000","last_became_peered":"2024-04-26T21:09:49.206907+0000","last_unstale":"2024-04-26T21:09:49.210284+0000","last_undegraded":"2024-04-26T21:09:49.210284+0000","last_fullsized":"2024-04-26T21:09:49.210284+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:18.880151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.247937+0000","last_change":"2024-04-26T21:09:48.169351+0000","last_active":"2024-04-26T21:09:50.247937+0000","last_peered":"2024-04-26T21:09:50.247937+0000","last_clean":"2024-04-26T21:09:50.247937+0000","last_became_active":"2024-04-26T21:09:48.168677+0000","last_became_peered":"2024-04-26T21:09:48.168677+0000","last_unstale":"2024-04-26T21:09:50.247937+0000","last_undegraded":"2024-04-26T21:09:50.247937+0000","last_fullsized":"2024-04-26T21:09:50.247937+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:29.553705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.224180+0000","last_change":"2024-04-26T21:09:49.181995+0000","last_active":"2024-04-26T21:09:50.224180+0000","last_peered":"2024-04-26T21:09:50.224180+0000","last_clean":"2024-04-26T21:09:50.224180+0000","last_became_active":"2024-04-26T21:09:49.181356+0000","last_became_peered":"2024-04-26T21:09:49.181356+0000","last_unstale":"2024-04-26T21:09:50.224180+0000","last_undegraded":"2024-04-26T21:09:50.224180+0000","last_fullsized":"2024-04-26T21:09:50.224180+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:53.758778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.247708+0000","last_change":"2024-04-26T21:09:48.159823+0000","last_active":"2024-04-26T21:09:50.247708+0000","last_peered":"2024-04-26T21:09:50.247708+0000","last_clean":"2024-04-26T21:09:50.247708+0000","last_became_active":"2024-04-26T21:09:48.158773+0000","last_became_peered":"2024-04-26T21:09:48.158773+0000","last_unstale":"2024-04-26T21:09:50.247708+0000","last_undegraded":"2024-04-26T21:09:50.247708+0000","last_fullsized":"2024-04-26T21:09:50.247708+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:11.900171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.204258+0000","last_change":"2024-04-26T21:09:49.193075+0000","last_active":"2024-04-26T21:09:49.204258+0000","last_peered":"2024-04-26T21:09:49.204258+0000","last_clean":"2024-04-26T21:09:49.204258+0000","last_became_active":"2024-04-26T21:09:49.192205+0000","last_became_peered":"2024-04-26T21:09:49.192205+0000","last_unstale":"2024-04-26T21:09:49.204258+0000","last_undegraded":"2024-04-26T21:09:49.204258+0000","last_fullsized":"2024-04-26T21:09:49.204258+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:39:33.860582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.248207+0000","last_change":"2024-04-26T21:09:48.162563+0000","last_active":"2024-04-26T21:09:50.248207+0000","last_peered":"2024-04-26T21:09:50.248207+0000","last_clean":"2024-04-26T21:09:50.248207+0000","last_became_active":"2024-04-26T21:09:48.160696+0000","last_became_peered":"2024-04-26T21:09:48.160696+0000","last_unstale":"2024-04-26T21:09:50.248207+0000","last_undegraded":"2024-04-26T21:09:50.248207+0000","last_fullsized":"2024-04-26T21:09:50.248207+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:11:19.364201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.184912+0000","last_change":"2024-04-26T21:09:49.182396+0000","last_active":"2024-04-26T21:09:49.184912+0000","last_peered":"2024-04-26T21:09:49.184912+0000","last_clean":"2024-04-26T21:09:49.184912+0000","last_became_active":"2024-04-26T21:09:49.181791+0000","last_became_peered":"2024-04-26T21:09:49.181791+0000","last_unstale":"2024-04-26T21:09:49.184912+0000","last_undegraded":"2024-04-26T21:09:49.184912+0000","last_fullsized":"2024-04-26T21:09:49.184912+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:55.773928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.193417+0000","last_change":"2024-04-26T21:09:48.165208+0000","last_active":"2024-04-26T21:09:50.193417+0000","last_peered":"2024-04-26T21:09:50.193417+0000","last_clean":"2024-04-26T21:09:50.193417+0000","last_became_active":"2024-04-26T21:09:48.164876+0000","last_became_peered":"2024-04-26T21:09:48.164876+0000","last_unstale":"2024-04-26T21:09:50.193417+0000","last_undegraded":"2024-04-26T21:09:50.193417+0000","last_fullsized":"2024-04-26T21:09:50.193417+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:07.546830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.204315+0000","last_change":"2024-04-26T21:09:49.194262+0000","last_active":"2024-04-26T21:09:49.204315+0000","last_peered":"2024-04-26T21:09:49.204315+0000","last_clean":"2024-04-26T21:09:49.204315+0000","last_became_active":"2024-04-26T21:09:49.193988+0000","last_became_peered":"2024-04-26T21:09:49.193988+0000","last_unstale":"2024-04-26T21:09:49.204315+0000","last_undegraded":"2024-04-26T21:09:49.204315+0000","last_fullsized":"2024-04-26T21:09:49.204315+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:14:50.294164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.199116+0000","last_change":"2024-04-26T21:09:48.173680+0000","last_active":"2024-04-26T21:09:50.199116+0000","last_peered":"2024-04-26T21:09:50.199116+0000","last_clean":"2024-04-26T21:09:50.199116+0000","last_became_active":"2024-04-26T21:09:48.170159+0000","last_became_peered":"2024-04-26T21:09:48.170159+0000","last_unstale":"2024-04-26T21:09:50.199116+0000","last_undegraded":"2024-04-26T21:09:50.199116+0000","last_fullsized":"2024-04-26T21:09:50.199116+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:39.478970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.204709+0000","last_change":"2024-04-26T21:09:49.196653+0000","last_active":"2024-04-26T21:09:49.204709+0000","last_peered":"2024-04-26T21:09:49.204709+0000","last_clean":"2024-04-26T21:09:49.204709+0000","last_became_active":"2024-04-26T21:09:49.196371+0000","last_became_peered":"2024-04-26T21:09:49.196371+0000","last_unstale":"2024-04-26T21:09:49.204709+0000","last_undegraded":"2024-04-26T21:09:49.204709+0000","last_fullsized":"2024-04-26T21:09:49.204709+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:42.942062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.247097+0000","last_change":"2024-04-26T21:09:48.167011+0000","last_active":"2024-04-26T21:09:50.247097+0000","last_peered":"2024-04-26T21:09:50.247097+0000","last_clean":"2024-04-26T21:09:50.247097+0000","last_became_active":"2024-04-26T21:09:48.164230+0000","last_became_peered":"2024-04-26T21:09:48.164230+0000","last_unstale":"2024-04-26T21:09:50.247097+0000","last_undegraded":"2024-04-26T21:09:50.247097+0000","last_fullsized":"2024-04-26T21:09:50.247097+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:28.162164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.197499+0000","last_change":"2024-04-26T21:09:49.200922+0000","last_active":"2024-04-26T21:09:50.197499+0000","last_peered":"2024-04-26T21:09:50.197499+0000","last_clean":"2024-04-26T21:09:50.197499+0000","last_became_active":"2024-04-26T21:09:49.200611+0000","last_became_peered":"2024-04-26T21:09:49.200611+0000","last_unstale":"2024-04-26T21:09:50.197499+0000","last_undegraded":"2024-04-26T21:09:50.197499+0000","last_fullsized":"2024-04-26T21:09:50.197499+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:05:37.764086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.189643+0000","last_change":"2024-04-26T21:09:48.175081+0000","last_active":"2024-04-26T21:09:49.189643+0000","last_peered":"2024-04-26T21:09:49.189643+0000","last_clean":"2024-04-26T21:09:49.189643+0000","last_became_active":"2024-04-26T21:09:48.174754+0000","last_became_peered":"2024-04-26T21:09:48.174754+0000","last_unstale":"2024-04-26T21:09:49.189643+0000","last_undegraded":"2024-04-26T21:09:49.189643+0000","last_fullsized":"2024-04-26T21:09:49.189643+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:20.091000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.204821+0000","last_change":"2024-04-26T21:09:49.198727+0000","last_active":"2024-04-26T21:09:49.204821+0000","last_peered":"2024-04-26T21:09:49.204821+0000","last_clean":"2024-04-26T21:09:49.204821+0000","last_became_active":"2024-04-26T21:09:49.198375+0000","last_became_peered":"2024-04-26T21:09:49.198375+0000","last_unstale":"2024-04-26T21:09:49.204821+0000","last_undegraded":"2024-04-26T21:09:49.204821+0000","last_fullsized":"2024-04-26T21:09:49.204821+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:57:29.177190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.224798+0000","last_change":"2024-04-26T21:09:48.175370+0000","last_active":"2024-04-26T21:09:50.224798+0000","last_peered":"2024-04-26T21:09:50.224798+0000","last_clean":"2024-04-26T21:09:50.224798+0000","last_became_active":"2024-04-26T21:09:48.175068+0000","last_became_peered":"2024-04-26T21:09:48.175068+0000","last_unstale":"2024-04-26T21:09:50.224798+0000","last_undegraded":"2024-04-26T21:09:50.224798+0000","last_fullsized":"2024-04-26T21:09:50.224798+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:07.546830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.212591+0000","last_change":"2024-04-26T21:09:49.209873+0000","last_active":"2024-04-26T21:09:49.212591+0000","last_peered":"2024-04-26T21:09:49.212591+0000","last_clean":"2024-04-26T21:09:49.212591+0000","last_became_active":"2024-04-26T21:09:49.209583+0000","last_became_peered":"2024-04-26T21:09:49.209583+0000","last_unstale":"2024-04-26T21:09:49.212591+0000","last_undegraded":"2024-04-26T21:09:49.212591+0000","last_fullsized":"2024-04-26T21:09:49.212591+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:07:20.080838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.221762+0000","last_change":"2024-04-26T21:09:48.168287+0000","last_active":"2024-04-26T21:09:50.221762+0000","last_peered":"2024-04-26T21:09:50.221762+0000","last_clean":"2024-04-26T21:09:50.221762+0000","last_became_active":"2024-04-26T21:09:48.167956+0000","last_became_peered":"2024-04-26T21:09:48.167956+0000","last_unstale":"2024-04-26T21:09:50.221762+0000","last_undegraded":"2024-04-26T21:09:50.221762+0000","last_fullsized":"2024-04-26T21:09:50.221762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:47:33.049064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.193099+0000","last_change":"2024-04-26T21:09:49.193102+0000","last_active":"2024-04-26T21:09:50.193099+0000","last_peered":"2024-04-26T21:09:50.193099+0000","last_clean":"2024-04-26T21:09:50.193099+0000","last_became_active":"2024-04-26T21:09:49.192772+0000","last_became_peered":"2024-04-26T21:09:49.192772+0000","last_unstale":"2024-04-26T21:09:50.193099+0000","last_undegraded":"2024-04-26T21:09:50.193099+0000","last_fullsized":"2024-04-26T21:09:50.193099+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:19.183103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.224294+0000","last_change":"2024-04-26T21:09:48.162408+0000","last_active":"2024-04-26T21:09:50.224294+0000","last_peered":"2024-04-26T21:09:50.224294+0000","last_clean":"2024-04-26T21:09:50.224294+0000","last_became_active":"2024-04-26T21:09:48.160352+0000","last_became_peered":"2024-04-26T21:09:48.160352+0000","last_unstale":"2024-04-26T21:09:50.224294+0000","last_undegraded":"2024-04-26T21:09:50.224294+0000","last_fullsized":"2024-04-26T21:09:50.224294+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:47:33.049064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.198843+0000","last_change":"2024-04-26T21:09:49.196308+0000","last_active":"2024-04-26T21:09:50.198843+0000","last_peered":"2024-04-26T21:09:50.198843+0000","last_clean":"2024-04-26T21:09:50.198843+0000","last_became_active":"2024-04-26T21:09:49.193660+0000","last_became_peered":"2024-04-26T21:09:49.193660+0000","last_unstale":"2024-04-26T21:09:50.198843+0000","last_undegraded":"2024-04-26T21:09:50.198843+0000","last_fullsized":"2024-04-26T21:09:50.198843+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:36.246542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.193638+0000","last_change":"2024-04-26T21:09:48.178997+0000","last_active":"2024-04-26T21:09:50.193638+0000","last_peered":"2024-04-26T21:09:50.193638+0000","last_clean":"2024-04-26T21:09:50.193638+0000","last_became_active":"2024-04-26T21:09:48.178694+0000","last_became_peered":"2024-04-26T21:09:48.178694+0000","last_unstale":"2024-04-26T21:09:50.193638+0000","last_undegraded":"2024-04-26T21:09:50.193638+0000","last_fullsized":"2024-04-26T21:09:50.193638+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:24:22.909294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.193305+0000","last_change":"2024-04-26T21:09:49.197803+0000","last_active":"2024-04-26T21:09:50.193305+0000","last_peered":"2024-04-26T21:09:50.193305+0000","last_clean":"2024-04-26T21:09:50.193305+0000","last_became_active":"2024-04-26T21:09:49.197547+0000","last_became_peered":"2024-04-26T21:09:49.197547+0000","last_unstale":"2024-04-26T21:09:50.193305+0000","last_undegraded":"2024-04-26T21:09:50.193305+0000","last_fullsized":"2024-04-26T21:09:50.193305+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:07:20.080838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.223801+0000","last_change":"2024-04-26T21:09:48.178326+0000","last_active":"2024-04-26T21:09:50.223801+0000","last_peered":"2024-04-26T21:09:50.223801+0000","last_clean":"2024-04-26T21:09:50.223801+0000","last_became_active":"2024-04-26T21:09:48.178106+0000","last_became_peered":"2024-04-26T21:09:48.178106+0000","last_unstale":"2024-04-26T21:09:50.223801+0000","last_undegraded":"2024-04-26T21:09:50.223801+0000","last_fullsized":"2024-04-26T21:09:50.223801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:04:02.163653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.203152+0000","last_change":"2024-04-26T21:09:49.195241+0000","last_active":"2024-04-26T21:09:49.203152+0000","last_peered":"2024-04-26T21:09:49.203152+0000","last_clean":"2024-04-26T21:09:49.203152+0000","last_became_active":"2024-04-26T21:09:49.194972+0000","last_became_peered":"2024-04-26T21:09:49.194972+0000","last_unstale":"2024-04-26T21:09:49.203152+0000","last_undegraded":"2024-04-26T21:09:49.203152+0000","last_fullsized":"2024-04-26T21:09:49.203152+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:29.134501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.247252+0000","last_change":"2024-04-26T21:09:48.176288+0000","last_active":"2024-04-26T21:09:50.247252+0000","last_peered":"2024-04-26T21:09:50.247252+0000","last_clean":"2024-04-26T21:09:50.247252+0000","last_became_active":"2024-04-26T21:09:48.176044+0000","last_became_peered":"2024-04-26T21:09:48.176044+0000","last_unstale":"2024-04-26T21:09:50.247252+0000","last_undegraded":"2024-04-26T21:09:50.247252+0000","last_fullsized":"2024-04-26T21:09:50.247252+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:08.043747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.246986+0000","last_change":"2024-04-26T21:09:49.194841+0000","last_active":"2024-04-26T21:09:50.246986+0000","last_peered":"2024-04-26T21:09:50.246986+0000","last_clean":"2024-04-26T21:09:50.246986+0000","last_became_active":"2024-04-26T21:09:49.194505+0000","last_became_peered":"2024-04-26T21:09:49.194505+0000","last_unstale":"2024-04-26T21:09:50.246986+0000","last_undegraded":"2024-04-26T21:09:50.246986+0000","last_fullsized":"2024-04-26T21:09:50.246986+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:36.246542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-26T21:09:49.188004+0000","last_change":"2024-04-26T21:09:48.173297+0000","last_active":"2024-04-26T21:09:49.188004+0000","last_peered":"2024-04-26T21:09:49.188004+0000","last_clean":"2024-04-26T21:09:49.188004+0000","last_became_active":"2024-04-26T21:09:48.172983+0000","last_became_peered":"2024-04-26T21:09:48.172983+0000","last_unstale":"2024-04-26T21:09:49.188004+0000","last_undegraded":"2024-04-26T21:09:49.188004+0000","last_fullsized":"2024-04-26T21:09:49.188004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:07.546830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.197819+0000","last_change":"2024-04-26T21:09:49.210602+0000","last_active":"2024-04-26T21:09:50.197819+0000","last_peered":"2024-04-26T21:09:50.197819+0000","last_clean":"2024-04-26T21:09:50.197819+0000","last_became_active":"2024-04-26T21:09:49.210229+0000","last_became_peered":"2024-04-26T21:09:49.210229+0000","last_unstale":"2024-04-26T21:09:50.197819+0000","last_undegraded":"2024-04-26T21:09:50.197819+0000","last_fullsized":"2024-04-26T21:09:50.197819+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:48.127676+0000","last_clean_scrub_stamp":"2024-04-26T21:09:48.127676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:55:08.786662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.198357+0000","last_change":"2024-04-26T21:09:48.173259+0000","last_active":"2024-04-26T21:09:50.198357+0000","last_peered":"2024-04-26T21:09:50.198357+0000","last_clean":"2024-04-26T21:09:50.198357+0000","last_became_active":"2024-04-26T21:09:48.172960+0000","last_became_peered":"2024-04-26T21:09:48.172960+0000","last_unstale":"2024-04-26T21:09:50.198357+0000","last_undegraded":"2024-04-26T21:09:50.198357+0000","last_fullsized":"2024-04-26T21:09:50.198357+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:09:47.112651+0000","last_clean_scrub_stamp":"2024-04-26T21:09:47.112651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:29.553705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":193,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T21:09:50.198056+0000","last_change":"2024-04-26T21:08:33.504911+0000","last_active":"2024-04-26T21:09:50.198056+0000","last_peered":"2024-04-26T21:09:50.198056+0000","last_clean":"2024-04-26T21:09:50.198056+0000","last_became_active":"2024-04-26T21:08:33.503667+0000","last_became_peered":"2024-04-26T21:08:33.503667+0000","last_unstale":"2024-04-26T21:09:50.198056+0000","last_undegraded":"2024-04-26T21:09:50.198056+0000","last_fullsized":"2024-04-26T21:09:50.198056+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:32.013907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:32.013907+0000","last_clean_scrub_stamp":"2024-04-26T21:08:32.013907+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:09.596153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":735437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":380928,"data_stored":145595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":145651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":145083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":146613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":376832,"data_stored":145049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31560,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340280,"statfs":{"total":96636764160,"available":96604446720,"internally_reserved":0,"allocated":389120,"data_stored":146613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":966656,"data_stored":735347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:09:54.196 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:09:54.196 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T21:09:54.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.460+0000 7f05d1000640 1 Processor -- start 2024-04-26T21:09:54.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.460+0000 7f05d1000640 1 -- start start 2024-04-26T21:09:54.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.462+0000 7f05d1000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f05cc0b5d30 0x7f05cc0b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:54.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.462+0000 7f05d1000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05cc0b42c0 0x7f05cc0b6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:54.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.462+0000 7f05d1000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f05cc0b6650 0x7f05cc06e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:54.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.462+0000 7f05d1000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f05cc073700 con 0x7f05cc0b6650 2024-04-26T21:09:54.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.462+0000 7f05d1000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f05cc06e420 con 0x7f05cc0b5d30 2024-04-26T21:09:54.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.462+0000 7f05d1000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f05cc146c00 con 0x7f05cc0b42c0 2024-04-26T21:09:54.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.463+0000 7f05ca800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f05cc0b6650 0x7f05cc06e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:54.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.463+0000 7f05ca800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f05cc0b6650 0x7f05cc06e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51108/0 (socket says 172.21.15.26:51108) 2024-04-26T21:09:54.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.463+0000 7f05ca800640 1 -- 172.21.15.26:0/315588702 learned_addr learned my addr 172.21.15.26:0/315588702 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:54.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.463+0000 7f05c9e00640 1 --2- 172.21.15.26:0/315588702 >> v2:172.21.15.86:3300/0 conn(0x7f05cc0b5d30 0x7f05cc0b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:54.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.463+0000 7f05c9400640 1 --2- 172.21.15.26:0/315588702 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05cc0b42c0 0x7f05cc0b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.463+0000 7f05ca800640 1 -- 172.21.15.26:0/315588702 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05cc0b42c0 msgr2=0x7f05cc0b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.463+0000 7f05ca800640 1 --2- 172.21.15.26:0/315588702 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05cc0b42c0 0x7f05cc0b6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.463+0000 7f05ca800640 1 -- 172.21.15.26:0/315588702 >> v2:172.21.15.86:3300/0 conn(0x7f05cc0b5d30 msgr2=0x7f05cc0b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.463+0000 7f05ca800640 1 --2- 172.21.15.26:0/315588702 >> v2:172.21.15.86:3300/0 conn(0x7f05cc0b5d30 0x7f05cc0b3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.463+0000 7f05ca800640 1 -- 172.21.15.26:0/315588702 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05cc072e60 con 0x7f05cc0b6650 2024-04-26T21:09:54.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.464+0000 7f05ca800640 1 --2- 172.21.15.26:0/315588702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05cc0b6650 0x7f05cc06e8b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0700b0 tx=0x7f05c402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d8a5ecc18bab67 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:54.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.465+0000 7f05c8a00640 1 -- 172.21.15.26:0/315588702 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c4036070 con 0x7f05cc0b6650 2024-04-26T21:09:54.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.465+0000 7f05c8a00640 1 -- 172.21.15.26:0/315588702 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c4031600 con 0x7f05cc0b6650 2024-04-26T21:09:54.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.465+0000 7f05c8a00640 1 -- 172.21.15.26:0/315588702 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c40319d0 con 0x7f05cc0b6650 2024-04-26T21:09:54.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.465+0000 7f05d1000640 1 -- 172.21.15.26:0/315588702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05cc0b6650 msgr2=0x7f05cc06e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:54.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.465+0000 7f05d1000640 1 --2- 172.21.15.26:0/315588702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05cc0b6650 0x7f05cc06e8b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0700b0 tx=0x7f05c402a6c0 comp rx=0 tx=0).stop 2024-04-26T21:09:54.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.466+0000 7f05d1000640 1 -- 172.21.15.26:0/315588702 shutdown_connections 2024-04-26T21:09:54.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.466+0000 7f05d1000640 1 --2- 172.21.15.26:0/315588702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05cc0b6650 0x7f05cc06e8b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:54.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.466+0000 7f05d1000640 1 --2- 172.21.15.26:0/315588702 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05cc0b42c0 0x7f05cc0b6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:54.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.466+0000 7f05d1000640 1 --2- 172.21.15.26:0/315588702 >> v2:172.21.15.86:3300/0 conn(0x7f05cc0b5d30 0x7f05cc0b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:54.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.466+0000 7f05d1000640 1 -- 172.21.15.26:0/315588702 >> 172.21.15.26:0/315588702 conn(0x7f05cc0b2a80 msgr2=0x7f05cc0b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:54.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.466+0000 7f05d1000640 1 -- 172.21.15.26:0/315588702 shutdown_connections 2024-04-26T21:09:54.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.466+0000 7f05d1000640 1 -- 172.21.15.26:0/315588702 wait complete. 2024-04-26T21:09:54.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.468+0000 7f05d1000640 1 Processor -- start 2024-04-26T21:09:54.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.468+0000 7f05d1000640 1 -- start start 2024-04-26T21:09:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.469+0000 7f05d1000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05cc0b42c0 0x7f05cc0ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.469+0000 7f05d1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05cc0b5d30 0x7f05cc0b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.469+0000 7f05d1000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05cc0b6650 0x7f05cc0ae850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.469+0000 7f05d1000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc072a80 con 0x7f05cc0b6650 2024-04-26T21:09:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.469+0000 7f05d1000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc073420 con 0x7f05cc0b5d30 2024-04-26T21:09:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.469+0000 7f05d1000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f05cc072c00 con 0x7f05cc0b42c0 2024-04-26T21:09:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.469+0000 7f05ca800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05cc0b6650 0x7f05cc0ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.469+0000 7f05ca800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05cc0b6650 0x7f05cc0ae850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51118/0 (socket says 172.21.15.26:51118) 2024-04-26T21:09:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.469+0000 7f05ca800640 1 -- 172.21.15.26:0/2266399327 learned_addr learned my addr 172.21.15.26:0/2266399327 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.469+0000 7f05c9e00640 1 --2- 172.21.15.26:0/2266399327 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05cc0b42c0 0x7f05cc0ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:54.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.469+0000 7f05c9400640 1 --2- 172.21.15.26:0/2266399327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05cc0b5d30 0x7f05cc0b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:54.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.469+0000 7f05ca800640 1 -- 172.21.15.26:0/2266399327 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05cc0b42c0 msgr2=0x7f05cc0ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:54.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.469+0000 7f05ca800640 1 --2- 172.21.15.26:0/2266399327 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05cc0b42c0 0x7f05cc0ce8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:54.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.469+0000 7f05ca800640 1 -- 172.21.15.26:0/2266399327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05cc0b5d30 msgr2=0x7f05cc0b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:54.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.469+0000 7f05ca800640 1 --2- 172.21.15.26:0/2266399327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05cc0b5d30 0x7f05cc0b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:54.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.469+0000 7f05ca800640 1 -- 172.21.15.26:0/2266399327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05cc0b0890 con 0x7f05cc0b6650 2024-04-26T21:09:54.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.469+0000 7f05c9400640 1 --2- 172.21.15.26:0/2266399327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05cc0b5d30 0x7f05cc0b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:54.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.470+0000 7f05ca800640 1 --2- 172.21.15.26:0/2266399327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05cc0b6650 0x7f05cc0ae850 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f05c4009f70 tx=0x7f05c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:54.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.470+0000 7f05c2000640 1 -- 172.21.15.26:0/2266399327 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c403f020 con 0x7f05cc0b6650 2024-04-26T21:09:54.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.470+0000 7f05c2000640 1 -- 172.21.15.26:0/2266399327 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c4037040 con 0x7f05cc0b6650 2024-04-26T21:09:54.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.470+0000 7f05d1000640 1 -- 172.21.15.26:0/2266399327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05cc0b0b70 con 0x7f05cc0b6650 2024-04-26T21:09:54.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.470+0000 7f05c2000640 1 -- 172.21.15.26:0/2266399327 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c4036070 con 0x7f05cc0b6650 2024-04-26T21:09:54.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.470+0000 7f05d1000640 1 -- 172.21.15.26:0/2266399327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05cc0b1410 con 0x7f05cc0b6650 2024-04-26T21:09:54.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.472+0000 7f05d1000640 1 -- 172.21.15.26:0/2266399327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05cc0b0f60 con 0x7f05cc0b6650 2024-04-26T21:09:54.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.473+0000 7f05c2000640 1 -- 172.21.15.26:0/2266399327 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f05c4042020 con 0x7f05cc0b6650 2024-04-26T21:09:54.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.473+0000 7f05c2000640 1 --2- 172.21.15.26:0/2266399327 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05ac06c8f0 0x7f05ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:54.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.474+0000 7f05c2000640 1 -- 172.21.15.26:0/2266399327 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f05c40ac0f0 con 0x7f05cc0b6650 2024-04-26T21:09:54.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.474+0000 7f05c9e00640 1 --2- 172.21.15.26:0/2266399327 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05ac06c8f0 0x7f05ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:54.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.474+0000 7f05c9e00640 1 --2- 172.21.15.26:0/2266399327 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05ac06c8f0 0x7f05ac06edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f05bc006fd0 tx=0x7f05bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:54.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.478+0000 7f05c2000640 1 -- 172.21.15.26:0/2266399327 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f05c4049030 con 0x7f05cc0b6650 2024-04-26T21:09:54.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.757+0000 7f05d1000640 1 -- 172.21.15.26:0/2266399327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f05cc142c90 con 0x7f05cc0b6650 2024-04-26T21:09:54.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.762+0000 7f05c2000640 1 -- 172.21.15.26:0/2266399327 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f05cc142c90 con 0x7f05cc0b6650 2024-04-26T21:09:54.763 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-04-26T21:09:54.763 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCdFyxmV7X3ChAArBm2KV2Tw+YKfYxawc6g2A== 2024-04-26T21:09:54.763 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-04-26T21:09:54.763 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-04-26T21:09:54.763 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T21:09:54.763 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-04-26T21:09:54.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.765+0000 7f05d1000640 1 -- 172.21.15.26:0/2266399327 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05ac06c8f0 msgr2=0x7f05ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:54.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.765+0000 7f05d1000640 1 --2- 172.21.15.26:0/2266399327 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05ac06c8f0 0x7f05ac06edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f05bc006fd0 tx=0x7f05bc008040 comp rx=0 tx=0).stop 2024-04-26T21:09:54.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.765+0000 7f05d1000640 1 -- 172.21.15.26:0/2266399327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05cc0b6650 msgr2=0x7f05cc0ae850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:54.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.765+0000 7f05d1000640 1 --2- 172.21.15.26:0/2266399327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05cc0b6650 0x7f05cc0ae850 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f05c4009f70 tx=0x7f05c4004a40 comp rx=0 tx=0).stop 2024-04-26T21:09:54.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.765+0000 7f05d1000640 1 -- 172.21.15.26:0/2266399327 shutdown_connections 2024-04-26T21:09:54.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.765+0000 7f05d1000640 1 --2- 172.21.15.26:0/2266399327 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05ac06c8f0 0x7f05ac06edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:54.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.765+0000 7f05d1000640 1 --2- 172.21.15.26:0/2266399327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05cc0b6650 0x7f05cc0ae850 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:54.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.765+0000 7f05d1000640 1 --2- 172.21.15.26:0/2266399327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05cc0b5d30 0x7f05cc0b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:54.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.765+0000 7f05d1000640 1 --2- 172.21.15.26:0/2266399327 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05cc0b42c0 0x7f05cc0ce8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:54.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.765+0000 7f05d1000640 1 -- 172.21.15.26:0/2266399327 >> 172.21.15.26:0/2266399327 conn(0x7f05cc0b2a80 msgr2=0x7f05cc143080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:54.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.765+0000 7f05d1000640 1 -- 172.21.15.26:0/2266399327 shutdown_connections 2024-04-26T21:09:54.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.765+0000 7f05d1000640 1 -- 172.21.15.26:0/2266399327 wait complete. 2024-04-26T21:09:54.785 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:54.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.992+0000 7ff476600640 1 Processor -- start 2024-04-26T21:09:54.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.992+0000 7ff476600640 1 -- start start 2024-04-26T21:09:54.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.994+0000 7ff476600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff47006f340 0x7ff47015faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:54.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.994+0000 7ff476600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff47015ffe0 0x7ff4700d11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:54.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.994+0000 7ff476600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff47006f7d0 con 0x7ff470073470 2024-04-26T21:09:54.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.994+0000 7ff476600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff470073f60 con 0x7ff47006f340 2024-04-26T21:09:54.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.994+0000 7ff476600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff47006fab0 con 0x7ff47015ffe0 2024-04-26T21:09:54.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.995+0000 7ff467e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff47006f340 0x7ff47015faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:54.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.995+0000 7ff467e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff47006f340 0x7ff47015faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.26:44522/0 (socket says 172.21.15.26:44522) 2024-04-26T21:09:54.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.995+0000 7ff467e00640 1 -- 172.21.15.26:0/1772550997 learned_addr learned my addr 172.21.15.26:0/1772550997 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:54.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.995+0000 7ff475c00640 1 --2- 172.21.15.26:0/1772550997 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff47015ffe0 0x7ff4700d11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:54.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.995+0000 7ff467e00640 1 -- 172.21.15.26:0/1772550997 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff47015ffe0 msgr2=0x7ff4700d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:54.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.995+0000 7ff467e00640 1 --2- 172.21.15.26:0/1772550997 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff47015ffe0 0x7ff4700d11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:54.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.995+0000 7ff467e00640 1 -- 172.21.15.26:0/1772550997 >> v1:172.21.15.86:6789/0 conn(0x7ff470073470 legacy=0x7ff470073850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:54.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.995+0000 7ff467e00640 1 -- 172.21.15.26:0/1772550997 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff47006fd90 con 0x7ff47006f340 2024-04-26T21:09:54.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.997+0000 7ff467e00640 1 --2- 172.21.15.26:0/1772550997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff47006f340 0x7ff47015faa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff458009870 tx=0x7ff45802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7bcefebab6595d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:54.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.997+0000 7ff46fe00640 1 -- 172.21.15.26:0/1772550997 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff458037070 con 0x7ff47006f340 2024-04-26T21:09:54.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.998+0000 7ff46fe00640 1 -- 172.21.15.26:0/1772550997 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff458032440 con 0x7ff47006f340 2024-04-26T21:09:54.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.998+0000 7ff46fe00640 1 -- 172.21.15.26:0/1772550997 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff458032810 con 0x7ff47006f340 2024-04-26T21:09:54.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.998+0000 7ff476600640 1 -- 172.21.15.26:0/1772550997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff47006f340 msgr2=0x7ff47015faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:54.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.998+0000 7ff476600640 1 --2- 172.21.15.26:0/1772550997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff47006f340 0x7ff47015faa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff458009870 tx=0x7ff45802a250 comp rx=0 tx=0).stop 2024-04-26T21:09:54.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.998+0000 7ff476600640 1 -- 172.21.15.26:0/1772550997 shutdown_connections 2024-04-26T21:09:55.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.998+0000 7ff476600640 1 --2- 172.21.15.26:0/1772550997 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff47015ffe0 0x7ff4700d11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.998+0000 7ff476600640 1 --2- 172.21.15.26:0/1772550997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff47006f340 0x7ff47015faa0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.998+0000 7ff476600640 1 -- 172.21.15.26:0/1772550997 >> 172.21.15.26:0/1772550997 conn(0x7ff47006c690 msgr2=0x7ff47006caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:55.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.998+0000 7ff476600640 1 -- 172.21.15.26:0/1772550997 shutdown_connections 2024-04-26T21:09:55.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:54.999+0000 7ff476600640 1 -- 172.21.15.26:0/1772550997 wait complete. 2024-04-26T21:09:55.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.000+0000 7ff476600640 1 Processor -- start 2024-04-26T21:09:55.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.000+0000 7ff476600640 1 -- start start 2024-04-26T21:09:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.001+0000 7ff476600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff470073470 0x7ff4701f5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.001+0000 7ff476600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff47015ffe0 0x7ff47015b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.001+0000 7ff476600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff47014fee0 0x7ff47015bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.001+0000 7ff476600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff4700b4d40 con 0x7ff47014fee0 2024-04-26T21:09:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.001+0000 7ff476600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff47015c4b0 con 0x7ff470073470 2024-04-26T21:09:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.001+0000 7ff476600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff4700b4ec0 con 0x7ff47015ffe0 2024-04-26T21:09:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.002+0000 7ff475c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff47014fee0 0x7ff47015bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.002+0000 7ff475c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff47014fee0 0x7ff47015bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51128/0 (socket says 172.21.15.26:51128) 2024-04-26T21:09:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.002+0000 7ff475c00640 1 -- 172.21.15.26:0/3616580935 learned_addr learned my addr 172.21.15.26:0/3616580935 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.002+0000 7ff475200640 1 --2- 172.21.15.26:0/3616580935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff470073470 0x7ff4701f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.002+0000 7ff467e00640 1 --2- 172.21.15.26:0/3616580935 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff47015ffe0 0x7ff47015b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.002+0000 7ff475c00640 1 -- 172.21.15.26:0/3616580935 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff47015ffe0 msgr2=0x7ff47015b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.002+0000 7ff475c00640 1 --2- 172.21.15.26:0/3616580935 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff47015ffe0 0x7ff47015b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.002+0000 7ff475c00640 1 -- 172.21.15.26:0/3616580935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff470073470 msgr2=0x7ff4701f5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.002+0000 7ff475c00640 1 --2- 172.21.15.26:0/3616580935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff470073470 0x7ff4701f5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.002+0000 7ff475c00640 1 -- 172.21.15.26:0/3616580935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4701578d0 con 0x7ff47014fee0 2024-04-26T21:09:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.003+0000 7ff475c00640 1 --2- 172.21.15.26:0/3616580935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff47014fee0 0x7ff47015bab0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff46800e7f0 tx=0x7ff46800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.003+0000 7ff46d600640 1 -- 172.21.15.26:0/3616580935 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff468017070 con 0x7ff47014fee0 2024-04-26T21:09:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.003+0000 7ff46d600640 1 -- 172.21.15.26:0/3616580935 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff46800c730 con 0x7ff47014fee0 2024-04-26T21:09:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.003+0000 7ff476600640 1 -- 172.21.15.26:0/3616580935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4701f5dc0 con 0x7ff47014fee0 2024-04-26T21:09:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.003+0000 7ff46d600640 1 -- 172.21.15.26:0/3616580935 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46800cb20 con 0x7ff47014fee0 2024-04-26T21:09:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.003+0000 7ff476600640 1 -- 172.21.15.26:0/3616580935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff470150830 con 0x7ff47014fee0 2024-04-26T21:09:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.004+0000 7ff476600640 1 -- 172.21.15.26:0/3616580935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff47006a700 con 0x7ff47014fee0 2024-04-26T21:09:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.006+0000 7ff46d600640 1 -- 172.21.15.26:0/3616580935 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff46801a070 con 0x7ff47014fee0 2024-04-26T21:09:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.007+0000 7ff46d600640 1 --2- 172.21.15.26:0/3616580935 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff44c06c8f0 0x7ff44c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.007+0000 7ff46d600640 1 -- 172.21.15.26:0/3616580935 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7ff46808ba40 con 0x7ff47014fee0 2024-04-26T21:09:55.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.007+0000 7ff475200640 1 --2- 172.21.15.26:0/3616580935 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff44c06c8f0 0x7ff44c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.008+0000 7ff475200640 1 --2- 172.21.15.26:0/3616580935 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff44c06c8f0 0x7ff44c06edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff47006cbe0 tx=0x7ff460009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.012+0000 7ff46d600640 1 -- 172.21.15.26:0/3616580935 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff4701f5dc0 con 0x7ff47014fee0 2024-04-26T21:09:55.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.258+0000 7ff476600640 1 -- 172.21.15.26:0/3616580935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff470159db0 con 0x7ff47014fee0 2024-04-26T21:09:55.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.259+0000 7ff46d600640 1 -- 172.21.15.26:0/3616580935 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7ff4701578d0 con 0x7ff47014fee0 2024-04-26T21:09:55.259 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:55.260 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2349160700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2349160700},{"type":"v1","addr":"172.21.15.26:6837","nonce":2349160700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4598,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2824191762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2824191762},{"type":"v1","addr":"172.21.15.86:6834","nonce":2824191762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/263746310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":263746310},{"type":"v1","addr":"172.21.15.86:6835","nonce":263746310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:09:48.622025+0000","modified":"2024-04-26T21:09:51.721103+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5058":{"gid":5058,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6836/2528180854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2528180854},{"type":"v1","addr":"172.21.15.26:6836","nonce":2528180854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5058,"qdb_cluster":[5058]},"id":2}]} 2024-04-26T21:09:55.260 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 17 2024-04-26T21:09:55.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.261+0000 7ff476600640 1 -- 172.21.15.26:0/3616580935 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff44c06c8f0 msgr2=0x7ff44c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.261+0000 7ff476600640 1 --2- 172.21.15.26:0/3616580935 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff44c06c8f0 0x7ff44c06edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff47006cbe0 tx=0x7ff460009040 comp rx=0 tx=0).stop 2024-04-26T21:09:55.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.261+0000 7ff476600640 1 -- 172.21.15.26:0/3616580935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff47014fee0 msgr2=0x7ff47015bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.261+0000 7ff476600640 1 --2- 172.21.15.26:0/3616580935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff47014fee0 0x7ff47015bab0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff46800e7f0 tx=0x7ff46800ecb0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.262+0000 7ff476600640 1 -- 172.21.15.26:0/3616580935 shutdown_connections 2024-04-26T21:09:55.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.262+0000 7ff476600640 1 --2- 172.21.15.26:0/3616580935 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff44c06c8f0 0x7ff44c06edb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.262+0000 7ff476600640 1 --2- 172.21.15.26:0/3616580935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff47014fee0 0x7ff47015bab0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.262+0000 7ff476600640 1 --2- 172.21.15.26:0/3616580935 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff47015ffe0 0x7ff47015b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.262+0000 7ff476600640 1 --2- 172.21.15.26:0/3616580935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff470073470 0x7ff4701f5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.262+0000 7ff476600640 1 -- 172.21.15.26:0/3616580935 >> 172.21.15.26:0/3616580935 conn(0x7ff47006c690 msgr2=0x7ff4700af260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:55.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.262+0000 7ff476600640 1 -- 172.21.15.26:0/3616580935 shutdown_connections 2024-04-26T21:09:55.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.262+0000 7ff476600640 1 -- 172.21.15.26:0/3616580935 wait complete. 2024-04-26T21:09:55.684 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T21:09:48.622025+0000', 'modified': '2024-04-26T21:09:51.721103+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5058}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5058': {'gid': 5058, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.26:6836/2528180854', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6835', 'nonce': 2528180854}, {'type': 'v1', 'addr': '172.21.15.26:6836', 'nonce': 2528180854}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5058, 'qdb_cluster': [5058]} 2024-04-26T21:09:55.684 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T21:09:55.685 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T21:09:55.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.904+0000 7fa65b200640 1 Processor -- start 2024-04-26T21:09:55.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.904+0000 7fa65b200640 1 -- start start 2024-04-26T21:09:55.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.906+0000 7fa65b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa65c073470 0x7fa65c073850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.906+0000 7fa65b200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa65c06f7d0 con 0x7fa65c0b4d40 2024-04-26T21:09:55.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.906+0000 7fa65b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa65c06fab0 con 0x7fa65c06f340 2024-04-26T21:09:55.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.906+0000 7fa65b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa65c06fd90 con 0x7fa65c073470 2024-04-26T21:09:55.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.906+0000 7fa65a800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa65c0b4d40 0x7fa65c0d11f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50192/0 (socket says 172.21.15.26:50192) 2024-04-26T21:09:55.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.906+0000 7fa65a800640 1 -- 172.21.15.26:0/2185933840 learned_addr learned my addr 172.21.15.26:0/2185933840 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:55.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.907+0000 7fa659e00640 1 --2- 172.21.15.26:0/2185933840 >> v2:172.21.15.86:3300/0 conn(0x7fa65c073470 0x7fa65c073850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.907+0000 7fa659400640 1 -- 172.21.15.26:0/2185933840 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 702405231 0 0) 0x7fa65c06f7d0 con 0x7fa65c0b4d40 2024-04-26T21:09:55.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.907+0000 7fa659400640 1 -- 172.21.15.26:0/2185933840 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa640003520 con 0x7fa65c0b4d40 2024-04-26T21:09:55.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.907+0000 7fa659400640 1 -- 172.21.15.26:0/2185933840 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1923873129 0 0) 0x7fa640003520 con 0x7fa65c0b4d40 2024-04-26T21:09:55.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.907+0000 7fa659400640 1 -- 172.21.15.26:0/2185933840 >> v2:172.21.15.86:3300/0 conn(0x7fa65c073470 msgr2=0x7fa65c073850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:55.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.907+0000 7fa659400640 1 --2- 172.21.15.26:0/2185933840 >> v2:172.21.15.86:3300/0 conn(0x7fa65c073470 0x7fa65c073850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.907+0000 7fa659400640 1 -- 172.21.15.26:0/2185933840 >> v1:172.21.15.86:6789/0 conn(0x7fa65c06f340 legacy=0x7fa65c15faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.907+0000 7fa659400640 1 -- 172.21.15.26:0/2185933840 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65c070070 con 0x7fa65c0b4d40 2024-04-26T21:09:55.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.908+0000 7fa659400640 1 -- 172.21.15.26:0/2185933840 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7fa6540019f0 con 0x7fa65c0b4d40 2024-04-26T21:09:55.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.908+0000 7fa659400640 1 -- 172.21.15.26:0/2185933840 >> v1:172.21.15.26:6789/0 conn(0x7fa65c0b4d40 legacy=0x7fa65c0d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.908+0000 7fa659400640 1 --2- 172.21.15.26:0/2185933840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa640003a00 0x7fa640003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.908+0000 7fa659400640 1 --2- 172.21.15.26:0/2185933840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa640004800 0x7fa640004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.908+0000 7fa659400640 1 --2- 172.21.15.26:0/2185933840 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6400051c0 0x7fa64000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.908+0000 7fa659400640 1 -- 172.21.15.26:0/2185933840 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c06fd90 con 0x7fa640004800 2024-04-26T21:09:55.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.908+0000 7fa659400640 1 -- 172.21.15.26:0/2185933840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa64000c0c0 con 0x7fa640003a00 2024-04-26T21:09:55.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.908+0000 7fa659400640 1 -- 172.21.15.26:0/2185933840 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa64000c2c0 con 0x7fa6400051c0 2024-04-26T21:09:55.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.908+0000 7fa653e00640 1 --2- 172.21.15.26:0/2185933840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa640004800 0x7fa640004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.908+0000 7fa659e00640 1 --2- 172.21.15.26:0/2185933840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa640003a00 0x7fa640003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.908+0000 7fa65a800640 1 --2- 172.21.15.26:0/2185933840 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6400051c0 0x7fa64000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.909+0000 7fa653e00640 1 -- 172.21.15.26:0/2185933840 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6400051c0 msgr2=0x7fa64000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:55.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.909+0000 7fa653e00640 1 --2- 172.21.15.26:0/2185933840 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6400051c0 0x7fa64000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.909+0000 7fa653e00640 1 -- 172.21.15.26:0/2185933840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa640003a00 msgr2=0x7fa640003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:55.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.909+0000 7fa653e00640 1 --2- 172.21.15.26:0/2185933840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa640003a00 0x7fa640003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.909+0000 7fa653e00640 1 -- 172.21.15.26:0/2185933840 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa640003520 con 0x7fa640004800 2024-04-26T21:09:55.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.909+0000 7fa65a800640 1 --2- 172.21.15.26:0/2185933840 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6400051c0 0x7fa64000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:55.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.909+0000 7fa65a800640 1 -- 172.21.15.26:0/2185933840 reap_dead start 2024-04-26T21:09:55.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.910+0000 7fa653e00640 1 --2- 172.21.15.26:0/2185933840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa640004800 0x7fa640004c80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa644001750 tx=0x7fa640005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560c77fbc10201b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:55.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.910+0000 7fa659400640 1 -- 172.21.15.26:0/2185933840 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644001370 con 0x7fa640004800 2024-04-26T21:09:55.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.910+0000 7fa659400640 1 -- 172.21.15.26:0/2185933840 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6440052d0 con 0x7fa640004800 2024-04-26T21:09:55.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.910+0000 7fa659400640 1 -- 172.21.15.26:0/2185933840 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64402abc0 con 0x7fa640004800 2024-04-26T21:09:55.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.910+0000 7fa65b200640 1 -- 172.21.15.26:0/2185933840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa640004800 msgr2=0x7fa640004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.910+0000 7fa65b200640 1 --2- 172.21.15.26:0/2185933840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa640004800 0x7fa640004c80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa644001750 tx=0x7fa640005830 comp rx=0 tx=0).stop 2024-04-26T21:09:55.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.911+0000 7fa65b200640 1 -- 172.21.15.26:0/2185933840 shutdown_connections 2024-04-26T21:09:55.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.911+0000 7fa65b200640 1 --2- 172.21.15.26:0/2185933840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa640004800 0x7fa640004c80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.911+0000 7fa65b200640 1 -- 172.21.15.26:0/2185933840 >> 172.21.15.26:0/2185933840 conn(0x7fa65c06c690 msgr2=0x7fa65c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:55.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.911+0000 7fa65b200640 1 -- 172.21.15.26:0/2185933840 shutdown_connections 2024-04-26T21:09:55.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.911+0000 7fa65b200640 1 -- 172.21.15.26:0/2185933840 wait complete. 2024-04-26T21:09:55.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.913+0000 7fa65b200640 1 Processor -- start 2024-04-26T21:09:55.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.913+0000 7fa65b200640 1 -- start start 2024-04-26T21:09:55.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.914+0000 7fa65b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa640004800 0x7fa65c1402c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.914+0000 7fa65b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa65c15c900 0x7fa65c15cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.914+0000 7fa65b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa65c15b090 0x7fa65c15b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.914+0000 7fa65b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c0739d0 con 0x7fa640004800 2024-04-26T21:09:55.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.914+0000 7fa65b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c073850 con 0x7fa65c15b090 2024-04-26T21:09:55.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.914+0000 7fa65b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa65c073b50 con 0x7fa65c15c900 2024-04-26T21:09:55.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.914+0000 7fa659e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa640004800 0x7fa65c1402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.914+0000 7fa659e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa640004800 0x7fa65c1402c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51158/0 (socket says 172.21.15.26:51158) 2024-04-26T21:09:55.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.914+0000 7fa659e00640 1 -- 172.21.15.26:0/1963171559 learned_addr learned my addr 172.21.15.26:0/1963171559 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:55.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.914+0000 7fa65a800640 1 --2- 172.21.15.26:0/1963171559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa65c15b090 0x7fa65c15b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.914+0000 7fa653e00640 1 --2- 172.21.15.26:0/1963171559 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa65c15c900 0x7fa65c15cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.915+0000 7fa659e00640 1 -- 172.21.15.26:0/1963171559 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa65c15c900 msgr2=0x7fa65c15cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.915+0000 7fa659e00640 1 --2- 172.21.15.26:0/1963171559 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa65c15c900 0x7fa65c15cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.915+0000 7fa659e00640 1 -- 172.21.15.26:0/1963171559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa65c15b090 msgr2=0x7fa65c15b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.915+0000 7fa659e00640 1 --2- 172.21.15.26:0/1963171559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa65c15b090 0x7fa65c15b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.915+0000 7fa659e00640 1 -- 172.21.15.26:0/1963171559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65c14ddc0 con 0x7fa640004800 2024-04-26T21:09:55.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.915+0000 7fa659e00640 1 --2- 172.21.15.26:0/1963171559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa640004800 0x7fa65c1402c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa64c011700 tx=0x7fa64c011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:55.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.915+0000 7fa652000640 1 -- 172.21.15.26:0/1963171559 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c00f9c0 con 0x7fa640004800 2024-04-26T21:09:55.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.915+0000 7fa652000640 1 -- 172.21.15.26:0/1963171559 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa64c00fb60 con 0x7fa640004800 2024-04-26T21:09:55.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.915+0000 7fa652000640 1 -- 172.21.15.26:0/1963171559 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c015460 con 0x7fa640004800 2024-04-26T21:09:55.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.915+0000 7fa65b200640 1 -- 172.21.15.26:0/1963171559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa65c142d50 con 0x7fa640004800 2024-04-26T21:09:55.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.915+0000 7fa65b200640 1 -- 172.21.15.26:0/1963171559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa65c143310 con 0x7fa640004800 2024-04-26T21:09:55.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.918+0000 7fa652000640 1 -- 172.21.15.26:0/1963171559 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa64c015600 con 0x7fa640004800 2024-04-26T21:09:55.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.919+0000 7fa652000640 1 --2- 172.21.15.26:0/1963171559 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa63806c920 0x7fa63806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.919+0000 7fa652000640 1 -- 172.21.15.26:0/1963171559 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fa64c08c0f0 con 0x7fa640004800 2024-04-26T21:09:55.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.919+0000 7fa653e00640 1 --2- 172.21.15.26:0/1963171559 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa63806c920 0x7fa63806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.920+0000 7fa663da4740 1 Processor -- start 2024-04-26T21:09:55.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.920+0000 7fa663da4740 1 -- start start 2024-04-26T21:09:55.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.920+0000 7fa653e00640 1 --2- 172.21.15.26:0/1963171559 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa63806c920 0x7fa63806ede0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa644032000 tx=0x7fa64402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:55.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.921+0000 7fa663da4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703b27b0 0x5618703b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.921+0000 7fa663da4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618703eee80 0x5618703b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.921+0000 7fa663da4740 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5618703dd270 0x5618703dd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.921+0000 7fa663da4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561870316600 con 0x5618703b27b0 2024-04-26T21:09:55.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.921+0000 7fa663da4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5618702b3100 con 0x5618703eee80 2024-04-26T21:09:55.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.921+0000 7fa663da4740 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x5618703125c0 con 0x5618703dd270 2024-04-26T21:09:55.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.921+0000 7fa659e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703b27b0 0x5618703b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.921+0000 7fa659e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703b27b0 0x5618703b4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51160/0 (socket says 172.21.15.26:51160) 2024-04-26T21:09:55.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.921+0000 7fa659e00640 1 -- 172.21.15.26:0/1250310706 learned_addr learned my addr 172.21.15.26:0/1250310706 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:55.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.921+0000 7fa65a800640 1 --2- 172.21.15.26:0/1250310706 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5618703dd270 0x5618703dd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.921+0000 7fa653e00640 1 --2- 172.21.15.26:0/1250310706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618703eee80 0x5618703b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.921+0000 7fa659e00640 1 -- 172.21.15.26:0/1250310706 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5618703dd270 msgr2=0x5618703dd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:55.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.921+0000 7fa659e00640 1 --2- 172.21.15.26:0/1250310706 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5618703dd270 0x5618703dd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.921+0000 7fa659e00640 1 -- 172.21.15.26:0/1250310706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618703eee80 msgr2=0x5618703b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:55.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.921+0000 7fa659e00640 1 --2- 172.21.15.26:0/1250310706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618703eee80 0x5618703b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.921+0000 7fa659e00640 1 -- 172.21.15.26:0/1250310706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618700282d0 con 0x5618703b27b0 2024-04-26T21:09:55.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.921+0000 7fa65a800640 1 --2- 172.21.15.26:0/1250310706 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5618703dd270 0x5618703dd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:55.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.921+0000 7fa653e00640 1 --2- 172.21.15.26:0/1250310706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618703eee80 0x5618703b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:55.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.922+0000 7fa659e00640 1 --2- 172.21.15.26:0/1250310706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703b27b0 0x5618703b4910 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa64c059e50 tx=0x7fa64c059f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b930e4788d0e7f7b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:55.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.922+0000 7fa64aa00640 1 -- 172.21.15.26:0/1250310706 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c022040 con 0x5618703b27b0 2024-04-26T21:09:55.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.922+0000 7fa64aa00640 1 -- 172.21.15.26:0/1250310706 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa64c057f60 con 0x5618703b27b0 2024-04-26T21:09:55.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.922+0000 7fa64aa00640 1 -- 172.21.15.26:0/1250310706 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c056530 con 0x5618703b27b0 2024-04-26T21:09:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.923+0000 7fa663da4740 1 -- 172.21.15.26:0/1250310706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703b27b0 msgr2=0x5618703b4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.923+0000 7fa663da4740 1 --2- 172.21.15.26:0/1250310706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703b27b0 0x5618703b4910 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa64c059e50 tx=0x7fa64c059f80 comp rx=0 tx=0).stop 2024-04-26T21:09:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.923+0000 7fa663da4740 1 -- 172.21.15.26:0/1250310706 shutdown_connections 2024-04-26T21:09:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.923+0000 7fa663da4740 1 --2- 172.21.15.26:0/1250310706 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5618703dd270 0x5618703dd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.923+0000 7fa663da4740 1 --2- 172.21.15.26:0/1250310706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618703eee80 0x5618703b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.923+0000 7fa663da4740 1 --2- 172.21.15.26:0/1250310706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703b27b0 0x5618703b4910 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.923+0000 7fa663da4740 1 -- 172.21.15.26:0/1250310706 >> 172.21.15.26:0/1250310706 conn(0x561870346d10 msgr2=0x5618703f8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.923+0000 7fa663da4740 1 -- 172.21.15.26:0/1250310706 shutdown_connections 2024-04-26T21:09:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.923+0000 7fa663da4740 1 -- 172.21.15.26:0/1250310706 wait complete. 2024-04-26T21:09:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.924+0000 7fa663da4740 1 -- start start 2024-04-26T21:09:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.924+0000 7fa663da4740 1 Processor -- start 2024-04-26T21:09:55.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.925+0000 7fa663da4740 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5618703b27b0 0x5618703fa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.925+0000 7fa663da4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618703dd270 0x5618703c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.925+0000 7fa663da4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703eee80 0x5618703c9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.925+0000 7fa663da4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56187029df70 con 0x5618703eee80 2024-04-26T21:09:55.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.925+0000 7fa663da4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5618703dadc0 con 0x5618703dd270 2024-04-26T21:09:55.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.925+0000 7fa663da4740 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x5618703c88d0 con 0x5618703b27b0 2024-04-26T21:09:55.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.925+0000 7fa663da4740 10 client.0 _ll_register_callbacks cb 0x56187030c480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:09:55.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.925+0000 7fa65a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703eee80 0x5618703c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.925+0000 7fa65a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703eee80 0x5618703c9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51170/0 (socket says 172.21.15.26:51170) 2024-04-26T21:09:55.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.925+0000 7fa65a800640 1 -- 172.21.15.26:0/3402072585 learned_addr learned my addr 172.21.15.26:0/3402072585 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:55.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.926+0000 7fa659e00640 1 --2- 172.21.15.26:0/3402072585 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5618703b27b0 0x5618703fa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.926+0000 7fa653e00640 1 --2- 172.21.15.26:0/3402072585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618703dd270 0x5618703c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.926+0000 7fa65a800640 1 -- 172.21.15.26:0/3402072585 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5618703b27b0 msgr2=0x5618703fa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:55.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.926+0000 7fa65a800640 1 --2- 172.21.15.26:0/3402072585 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5618703b27b0 0x5618703fa340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.926+0000 7fa65a800640 1 -- 172.21.15.26:0/3402072585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618703dd270 msgr2=0x5618703c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:55.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.926+0000 7fa65a800640 1 --2- 172.21.15.26:0/3402072585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618703dd270 0x5618703c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.926+0000 7fa65a800640 1 -- 172.21.15.26:0/3402072585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618700f26b0 con 0x5618703eee80 2024-04-26T21:09:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.926+0000 7fa659e00640 1 --2- 172.21.15.26:0/3402072585 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5618703b27b0 0x5618703fa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.926+0000 7fa65a800640 1 --2- 172.21.15.26:0/3402072585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703eee80 0x5618703c9130 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa65c1535a0 tx=0x7fa654007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864a6468e2e4d4ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.926+0000 7fa63f400640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-26T21:09:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.926+0000 7fa63f400640 1 -- 172.21.15.26:0/3402072585 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654010990 con 0x5618703eee80 2024-04-26T21:09:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.926+0000 7fa63f400640 1 -- 172.21.15.26:0/3402072585 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fa654010b30 con 0x5618703eee80 2024-04-26T21:09:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.927+0000 7fa663da4740 1 -- 172.21.15.26:0/3402072585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa64c057f60 con 0x5618703eee80 2024-04-26T21:09:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.927+0000 7fa63f400640 1 -- 172.21.15.26:0/3402072585 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa654021050 con 0x5618703eee80 2024-04-26T21:09:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.927+0000 7fa663da4740 10 client.5235 fetch_fsmap learned FSMap version 17 2024-04-26T21:09:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.927+0000 7fa663da4740 1 -- 172.21.15.26:0/3402072585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5618703ddae0 con 0x5618703eee80 2024-04-26T21:09:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.927+0000 7fa63f400640 1 -- 172.21.15.26:0/3402072585 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa64c057f60 con 0x5618703eee80 2024-04-26T21:09:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.927+0000 7fa63f400640 1 -- 172.21.15.26:0/3402072585 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7fa654024020 con 0x5618703eee80 2024-04-26T21:09:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.927+0000 7fa663da4740 10 client.5235 fetch_fsmap finished waiting for FSMap version 17 2024-04-26T21:09:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.927+0000 7fa663da4740 10 client.5235 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:09:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.927+0000 7fa663da4740 20 client.5235 populate_metadata read hostname 'smithi026' 2024-04-26T21:09:55.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.927+0000 7fa663da4740 1 --2- 172.21.15.26:0/3402072585 >> [v2:172.21.15.26:6835/2528180854,v1:172.21.15.26:6836/2528180854] conn(0x5618703c7850 0x5618703c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.927+0000 7fa663da4740 4 client.5235 mds_command: new command op to 5058 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:09:55.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.927+0000 7fa663da4740 1 -- 172.21.15.26:0/3402072585 --> [v2:172.21.15.26:6835/2528180854,v1:172.21.15.26:6836/2528180854] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561870324210 con 0x5618703c7850 2024-04-26T21:09:55.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.928+0000 7fa659e00640 1 --2- 172.21.15.26:0/3402072585 >> [v2:172.21.15.26:6835/2528180854,v1:172.21.15.26:6836/2528180854] conn(0x5618703c7850 0x5618703c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.928+0000 7fa659e00640 1 --2- 172.21.15.26:0/3402072585 >> [v2:172.21.15.26:6835/2528180854,v1:172.21.15.26:6836/2528180854] conn(0x5618703c7850 0x5618703c7c30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24e33cdff1d6e1e3 server_cookie=646dfaeecf3443b3 in_seq=0 out_seq=0 2024-04-26T21:09:55.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.928+0000 7fa63f400640 10 client.5235 ms_handle_connect on v2:172.21.15.26:6835/2528180854 2024-04-26T21:09:55.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.929+0000 7fa63f400640 1 -- 172.21.15.26:0/3402072585 <== mds.0 v2:172.21.15.26:6835/2528180854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561870324210 con 0x5618703c7850 2024-04-26T21:09:55.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.929+0000 7fa63f400640 10 client.5235 handle_command_reply: tid=0 2024-04-26T21:09:55.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.929+0000 7fa663da4740 1 client.5235 shutdown 2024-04-26T21:09:55.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.929+0000 7fa663da4740 1 -- 172.21.15.26:0/3402072585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703eee80 msgr2=0x5618703c9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.929+0000 7fa663da4740 1 --2- 172.21.15.26:0/3402072585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703eee80 0x5618703c9130 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa65c1535a0 tx=0x7fa654007710 comp rx=0 tx=0).stop 2024-04-26T21:09:55.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.929+0000 7fa663da4740 1 -- 172.21.15.26:0/3402072585 shutdown_connections 2024-04-26T21:09:55.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.929+0000 7fa663da4740 1 --2- 172.21.15.26:0/3402072585 >> [v2:172.21.15.26:6835/2528180854,v1:172.21.15.26:6836/2528180854] conn(0x5618703c7850 0x5618703c7c30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.929+0000 7fa663da4740 1 --2- 172.21.15.26:0/3402072585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703eee80 0x5618703c9130 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.929+0000 7fa663da4740 1 --2- 172.21.15.26:0/3402072585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618703dd270 0x5618703c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.929+0000 7fa663da4740 1 --2- 172.21.15.26:0/3402072585 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5618703b27b0 0x5618703fa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.929+0000 7fa663da4740 1 -- 172.21.15.26:0/3402072585 >> 172.21.15.26:0/3402072585 conn(0x561870346d10 msgr2=0x5618703db1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:55.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.930+0000 7fa63f400640 0 client.5235 ms_handle_reset on v2:172.21.15.26:6835/2528180854 2024-04-26T21:09:55.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.930+0000 7fa663da4740 1 -- 172.21.15.26:0/3402072585 shutdown_connections 2024-04-26T21:09:55.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.930+0000 7fa663da4740 1 -- 172.21.15.26:0/3402072585 wait complete. 2024-04-26T21:09:55.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.930+0000 7fa663da4740 20 client.5235 trim_cache size 0 max 16384 2024-04-26T21:09:55.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.950+0000 7fa663da4740 1 Processor -- start 2024-04-26T21:09:55.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.950+0000 7fa663da4740 1 -- start start 2024-04-26T21:09:55.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.951+0000 7fa663da4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703b27b0 0x5618703da4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.951+0000 7fa663da4740 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5618703dd270 0x5618703dff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.951+0000 7fa663da4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618703eee80 0x5618703d6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.951+0000 7fa663da4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c200f20 con 0x5618703b27b0 2024-04-26T21:09:55.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.951+0000 7fa663da4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c0af440 con 0x5618703eee80 2024-04-26T21:09:55.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.951+0000 7fa663da4740 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa65c2008d0 con 0x5618703dd270 2024-04-26T21:09:55.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.951+0000 7fa659e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703b27b0 0x5618703da4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.951+0000 7fa659e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703b27b0 0x5618703da4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51186/0 (socket says 172.21.15.26:51186) 2024-04-26T21:09:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.951+0000 7fa659e00640 1 -- 172.21.15.26:0/399824172 learned_addr learned my addr 172.21.15.26:0/399824172 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.952+0000 7fa65a800640 1 --2- 172.21.15.26:0/399824172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618703eee80 0x5618703d6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.952+0000 7fa653e00640 1 --2- 172.21.15.26:0/399824172 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5618703dd270 0x5618703dff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.952+0000 7fa659e00640 1 -- 172.21.15.26:0/399824172 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5618703dd270 msgr2=0x5618703dff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.952+0000 7fa659e00640 1 --2- 172.21.15.26:0/399824172 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5618703dd270 0x5618703dff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.952+0000 7fa659e00640 1 -- 172.21.15.26:0/399824172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618703eee80 msgr2=0x5618703d6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.952+0000 7fa659e00640 1 --2- 172.21.15.26:0/399824172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618703eee80 0x5618703d6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.952+0000 7fa659e00640 1 -- 172.21.15.26:0/399824172 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618703b0b10 con 0x5618703b27b0 2024-04-26T21:09:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.952+0000 7fa65a800640 1 --2- 172.21.15.26:0/399824172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618703eee80 0x5618703d6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.952+0000 7fa659e00640 1 --2- 172.21.15.26:0/399824172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703b27b0 0x5618703da4c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00f5b0 tx=0x7fa64c05c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b8d8c5b45f7656 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.952+0000 7fa64a000640 1 -- 172.21.15.26:0/399824172 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c089040 con 0x5618703b27b0 2024-04-26T21:09:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.952+0000 7fa64a000640 1 -- 172.21.15.26:0/399824172 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa64c076070 con 0x5618703b27b0 2024-04-26T21:09:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.952+0000 7fa64a000640 1 -- 172.21.15.26:0/399824172 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c022040 con 0x5618703b27b0 2024-04-26T21:09:55.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.953+0000 7fa663da4740 1 -- 172.21.15.26:0/399824172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703b27b0 msgr2=0x5618703da4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.953+0000 7fa663da4740 1 --2- 172.21.15.26:0/399824172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703b27b0 0x5618703da4c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00f5b0 tx=0x7fa64c05c000 comp rx=0 tx=0).stop 2024-04-26T21:09:55.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.953+0000 7fa663da4740 1 -- 172.21.15.26:0/399824172 shutdown_connections 2024-04-26T21:09:55.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.953+0000 7fa663da4740 1 --2- 172.21.15.26:0/399824172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618703eee80 0x5618703d6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.953+0000 7fa663da4740 1 --2- 172.21.15.26:0/399824172 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5618703dd270 0x5618703dff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.953+0000 7fa663da4740 1 --2- 172.21.15.26:0/399824172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703b27b0 0x5618703da4c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.953+0000 7fa663da4740 1 -- 172.21.15.26:0/399824172 >> 172.21.15.26:0/399824172 conn(0x561870346d10 msgr2=0x5618703f2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:55.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.953+0000 7fa663da4740 1 -- 172.21.15.26:0/399824172 shutdown_connections 2024-04-26T21:09:55.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.953+0000 7fa663da4740 1 -- 172.21.15.26:0/399824172 wait complete. 2024-04-26T21:09:55.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.954+0000 7fa663da4740 1 -- start start 2024-04-26T21:09:55.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.954+0000 7fa663da4740 1 Processor -- start 2024-04-26T21:09:55.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.955+0000 7fa663da4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703b27b0 0x56187045ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.955+0000 7fa663da4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618703dd270 0x5618703d6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.955+0000 7fa663da4740 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5618703eee80 0x5618703dff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.955+0000 7fa663da4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5618703dca80 con 0x5618703b27b0 2024-04-26T21:09:55.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.955+0000 7fa663da4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5618703c99d0 con 0x5618703dd270 2024-04-26T21:09:55.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.955+0000 7fa663da4740 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x5618703d96f0 con 0x5618703eee80 2024-04-26T21:09:55.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.955+0000 7fa663da4740 10 client.0 _ll_register_callbacks cb 0x7fa6540019f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:09:55.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.955+0000 7fa659e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703b27b0 0x56187045ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.955+0000 7fa659e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703b27b0 0x56187045ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51198/0 (socket says 172.21.15.26:51198) 2024-04-26T21:09:55.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.955+0000 7fa659e00640 1 -- 172.21.15.26:0/1561108677 learned_addr learned my addr 172.21.15.26:0/1561108677 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:55.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.956+0000 7fa659e00640 1 -- 172.21.15.26:0/1561108677 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5618703eee80 msgr2=0x5618703dff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:55.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.956+0000 7fa659e00640 1 --2- 172.21.15.26:0/1561108677 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5618703eee80 0x5618703dff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.956+0000 7fa659e00640 1 -- 172.21.15.26:0/1561108677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618703dd270 msgr2=0x5618703d6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:55.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.956+0000 7fa659e00640 1 --2- 172.21.15.26:0/1561108677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618703dd270 0x5618703d6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.956+0000 7fa659e00640 1 -- 172.21.15.26:0/1561108677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618703f14c0 con 0x5618703b27b0 2024-04-26T21:09:55.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.956+0000 7fa659e00640 1 --2- 172.21.15.26:0/1561108677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703b27b0 0x56187045ad30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa64c0116d0 tx=0x7fa64c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e2b29f1cf0b3d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:55.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.956+0000 7fa63f400640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-04-26T21:09:55.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.956+0000 7fa63f400640 1 -- 172.21.15.26:0/1561108677 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c022040 con 0x5618703b27b0 2024-04-26T21:09:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.956+0000 7fa63f400640 1 -- 172.21.15.26:0/1561108677 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fa64c07ade0 con 0x5618703b27b0 2024-04-26T21:09:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.956+0000 7fa663da4740 1 -- 172.21.15.26:0/1561108677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa64c076070 con 0x5618703b27b0 2024-04-26T21:09:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.957+0000 7fa63f400640 1 -- 172.21.15.26:0/1561108677 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa64c079370 con 0x5618703b27b0 2024-04-26T21:09:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.957+0000 7fa663da4740 10 client.5241 fetch_fsmap learned FSMap version 17 2024-04-26T21:09:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.957+0000 7fa663da4740 1 -- 172.21.15.26:0/1561108677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5618703f1dc0 con 0x5618703b27b0 2024-04-26T21:09:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.957+0000 7fa63f400640 1 -- 172.21.15.26:0/1561108677 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa64c076070 con 0x5618703b27b0 2024-04-26T21:09:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.957+0000 7fa63f400640 1 -- 172.21.15.26:0/1561108677 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7fa64c025020 con 0x5618703b27b0 2024-04-26T21:09:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.957+0000 7fa663da4740 10 client.5241 fetch_fsmap finished waiting for FSMap version 17 2024-04-26T21:09:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.957+0000 7fa663da4740 10 client.5241 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T21:09:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.957+0000 7fa663da4740 20 client.5241 populate_metadata read hostname 'smithi026' 2024-04-26T21:09:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.957+0000 7fa663da4740 1 --2- 172.21.15.26:0/1561108677 >> [v2:172.21.15.26:6835/2528180854,v1:172.21.15.26:6836/2528180854] conn(0x5618703d84c0 0x5618703d88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.957+0000 7fa663da4740 4 client.5241 mds_command: new command op to 5058 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:09:55.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.957+0000 7fa663da4740 1 -- 172.21.15.26:0/1561108677 --> [v2:172.21.15.26:6835/2528180854,v1:172.21.15.26:6836/2528180854] -- command(tid 0: {"prefix": "status"}) -- 0x56187031e080 con 0x5618703d84c0 2024-04-26T21:09:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.957+0000 7fa653e00640 1 --2- 172.21.15.26:0/1561108677 >> [v2:172.21.15.26:6835/2528180854,v1:172.21.15.26:6836/2528180854] conn(0x5618703d84c0 0x5618703d88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.958+0000 7fa653e00640 1 --2- 172.21.15.26:0/1561108677 >> [v2:172.21.15.26:6835/2528180854,v1:172.21.15.26:6836/2528180854] conn(0x5618703d84c0 0x5618703d88a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58c7325098551e6e server_cookie=53fd3b565c91c741 in_seq=0 out_seq=0 2024-04-26T21:09:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.958+0000 7fa63f400640 10 client.5241 ms_handle_connect on v2:172.21.15.26:6835/2528180854 2024-04-26T21:09:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.958+0000 7fa63f400640 1 -- 172.21.15.26:0/1561108677 <== mds.0 v2:172.21.15.26:6835/2528180854 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x56187031e080 con 0x5618703d84c0 2024-04-26T21:09:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.958+0000 7fa63f400640 10 client.5241 handle_command_reply: tid=0 2024-04-26T21:09:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.958+0000 7fa663da4740 1 client.5241 shutdown 2024-04-26T21:09:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.959+0000 7fa663da4740 1 -- 172.21.15.26:0/1561108677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703b27b0 msgr2=0x56187045ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.959+0000 7fa663da4740 1 --2- 172.21.15.26:0/1561108677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703b27b0 0x56187045ad30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa64c0116d0 tx=0x7fa64c014000 comp rx=0 tx=0).stop 2024-04-26T21:09:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.959+0000 7fa663da4740 1 -- 172.21.15.26:0/1561108677 shutdown_connections 2024-04-26T21:09:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.959+0000 7fa663da4740 1 --2- 172.21.15.26:0/1561108677 >> [v2:172.21.15.26:6835/2528180854,v1:172.21.15.26:6836/2528180854] conn(0x5618703d84c0 0x5618703d88a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.959+0000 7fa663da4740 1 --2- 172.21.15.26:0/1561108677 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5618703eee80 0x5618703dff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.959+0000 7fa663da4740 1 --2- 172.21.15.26:0/1561108677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5618703dd270 0x5618703d6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.959+0000 7fa663da4740 1 --2- 172.21.15.26:0/1561108677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5618703b27b0 0x56187045ad30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.959+0000 7fa63f400640 0 client.5241 ms_handle_reset on v2:172.21.15.26:6835/2528180854 2024-04-26T21:09:55.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.959+0000 7fa663da4740 1 -- 172.21.15.26:0/1561108677 >> 172.21.15.26:0/1561108677 conn(0x561870346d10 msgr2=0x5618703da620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:55.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.959+0000 7fa663da4740 1 -- 172.21.15.26:0/1561108677 shutdown_connections 2024-04-26T21:09:55.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.959+0000 7fa663da4740 1 -- 172.21.15.26:0/1561108677 wait complete. 2024-04-26T21:09:55.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.959+0000 7fa663da4740 20 client.5241 trim_cache size 0 max 16384 2024-04-26T21:09:55.964 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-26T21:09:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "eaf60df9-83a0-4283-ab2c-83d2c02a2067", 2024-04-26T21:09:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-04-26T21:09:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5058, 2024-04-26T21:09:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-04-26T21:09:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-04-26T21:09:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-04-26T21:09:55.965 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 6.7723322599999998, 2024-04-26T21:09:55.965 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 17, 2024-04-26T21:09:55.965 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 34, 2024-04-26T21:09:55.965 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 34, 2024-04-26T21:09:55.965 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 15.633303936000001 2024-04-26T21:09:55.965 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-26T21:09:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.960+0000 7fa65b200640 1 -- 172.21.15.26:0/1963171559 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa63806c920 msgr2=0x7fa63806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.960+0000 7fa65b200640 1 --2- 172.21.15.26:0/1963171559 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa63806c920 0x7fa63806ede0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa644032000 tx=0x7fa64402e000 comp rx=0 tx=0).stop 2024-04-26T21:09:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.960+0000 7fa65b200640 1 -- 172.21.15.26:0/1963171559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa640004800 msgr2=0x7fa65c1402c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.960+0000 7fa65b200640 1 --2- 172.21.15.26:0/1963171559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa640004800 0x7fa65c1402c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa64c011700 tx=0x7fa64c011bc0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.960+0000 7fa65b200640 1 -- 172.21.15.26:0/1963171559 shutdown_connections 2024-04-26T21:09:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.960+0000 7fa65b200640 1 --2- 172.21.15.26:0/1963171559 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa63806c920 0x7fa63806ede0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.960+0000 7fa65b200640 1 --2- 172.21.15.26:0/1963171559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa65c15b090 0x7fa65c15b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.960+0000 7fa65b200640 1 --2- 172.21.15.26:0/1963171559 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa65c15c900 0x7fa65c15cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.960+0000 7fa65b200640 1 --2- 172.21.15.26:0/1963171559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa640004800 0x7fa65c1402c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.960+0000 7fa65b200640 1 -- 172.21.15.26:0/1963171559 >> 172.21.15.26:0/1963171559 conn(0x7fa65c06c690 msgr2=0x7fa65c151fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.960+0000 7fa65b200640 1 -- 172.21.15.26:0/1963171559 shutdown_connections 2024-04-26T21:09:55.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:55.960+0000 7fa65b200640 1 -- 172.21.15.26:0/1963171559 wait complete. 2024-04-26T21:09:55.980 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T21:09:56.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.192+0000 7f24d1a00640 1 Processor -- start 2024-04-26T21:09:56.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.192+0000 7f24d1a00640 1 -- start start 2024-04-26T21:09:56.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.194+0000 7f24d1a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24cc162740 0x7f24cc06fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:56.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.194+0000 7f24d1a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f24cc160930 0x7f24cc06ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:56.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.194+0000 7f24d1a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24cc162b20 con 0x7f24cc0b2e30 2024-04-26T21:09:56.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.194+0000 7f24d1a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f24cc0cd750 con 0x7f24cc160930 2024-04-26T21:09:56.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.194+0000 7f24d1a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f24cc0cda30 con 0x7f24cc162740 2024-04-26T21:09:56.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.195+0000 7f24cbe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24cc162740 0x7f24cc06fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:56.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.195+0000 7f24cb400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f24cc160930 0x7f24cc06ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:56.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.195+0000 7f24cb400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f24cc160930 0x7f24cc06ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.26:44616/0 (socket says 172.21.15.26:44616) 2024-04-26T21:09:56.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.195+0000 7f24cb400640 1 -- 172.21.15.26:0/1000282880 learned_addr learned my addr 172.21.15.26:0/1000282880 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:56.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.195+0000 7f24cb400640 1 -- 172.21.15.26:0/1000282880 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24cc162740 msgr2=0x7f24cc06fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:56.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.195+0000 7f24cb400640 1 --2- 172.21.15.26:0/1000282880 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24cc162740 0x7f24cc06fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:56.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.195+0000 7f24cb400640 1 -- 172.21.15.26:0/1000282880 >> v1:172.21.15.86:6789/0 conn(0x7f24cc0b2e30 legacy=0x7f24cc0ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:56.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.195+0000 7f24cb400640 1 -- 172.21.15.26:0/1000282880 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24cc0cdd10 con 0x7f24cc160930 2024-04-26T21:09:56.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.197+0000 7f24cb400640 1 --2- 172.21.15.26:0/1000282880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24cc160930 0x7f24cc06ff80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f24bc009870 tx=0x7f24bc02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eadcccd660a8bc8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:56.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.197+0000 7f24caa00640 1 -- 172.21.15.26:0/1000282880 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc037070 con 0x7f24cc160930 2024-04-26T21:09:56.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.197+0000 7f24caa00640 1 -- 172.21.15.26:0/1000282880 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24bc032440 con 0x7f24cc160930 2024-04-26T21:09:56.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.197+0000 7f24caa00640 1 -- 172.21.15.26:0/1000282880 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc032810 con 0x7f24cc160930 2024-04-26T21:09:56.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.198+0000 7f24d1a00640 1 -- 172.21.15.26:0/1000282880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24cc160930 msgr2=0x7f24cc06ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:56.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.198+0000 7f24d1a00640 1 --2- 172.21.15.26:0/1000282880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24cc160930 0x7f24cc06ff80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f24bc009870 tx=0x7f24bc02a130 comp rx=0 tx=0).stop 2024-04-26T21:09:56.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.198+0000 7f24d1a00640 1 -- 172.21.15.26:0/1000282880 shutdown_connections 2024-04-26T21:09:56.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.198+0000 7f24d1a00640 1 --2- 172.21.15.26:0/1000282880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24cc160930 0x7f24cc06ff80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:56.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.198+0000 7f24d1a00640 1 --2- 172.21.15.26:0/1000282880 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24cc162740 0x7f24cc06fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:56.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.198+0000 7f24d1a00640 1 -- 172.21.15.26:0/1000282880 >> 172.21.15.26:0/1000282880 conn(0x7f24cc06c660 msgr2=0x7f24cc06ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:56.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.198+0000 7f24d1a00640 1 -- 172.21.15.26:0/1000282880 shutdown_connections 2024-04-26T21:09:56.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.198+0000 7f24d1a00640 1 -- 172.21.15.26:0/1000282880 wait complete. 2024-04-26T21:09:56.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.200+0000 7f24d1a00640 1 Processor -- start 2024-04-26T21:09:56.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.200+0000 7f24d1a00640 1 -- start start 2024-04-26T21:09:56.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.201+0000 7f24d1a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24cc0b2e30 0x7f24cc0c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:56.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.201+0000 7f24d1a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24cc160930 0x7f24cc0c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:56.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.201+0000 7f24d1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24cc162740 0x7f24cc155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:56.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.201+0000 7f24d1a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc0d0920 con 0x7f24cc160930 2024-04-26T21:09:56.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.201+0000 7f24d1a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc0ce2d0 con 0x7f24cc162740 2024-04-26T21:09:56.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.201+0000 7f24d1a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f24cc0d0aa0 con 0x7f24cc0b2e30 2024-04-26T21:09:56.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.201+0000 7f24cb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24cc160930 0x7f24cc0c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:56.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.201+0000 7f24cb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24cc160930 0x7f24cc0c7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51212/0 (socket says 172.21.15.26:51212) 2024-04-26T21:09:56.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.201+0000 7f24cb400640 1 -- 172.21.15.26:0/585237696 learned_addr learned my addr 172.21.15.26:0/585237696 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:56.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.201+0000 7f24cbe00640 1 --2- 172.21.15.26:0/585237696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24cc0b2e30 0x7f24cc0c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:56.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.201+0000 7f24cb400640 1 -- 172.21.15.26:0/585237696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24cc0b2e30 msgr2=0x7f24cc0c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:56.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.201+0000 7f24cb400640 1 --2- 172.21.15.26:0/585237696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24cc0b2e30 0x7f24cc0c6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:56.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.201+0000 7f24cb400640 1 -- 172.21.15.26:0/585237696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24cc162740 msgr2=0x7f24cc155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:56.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.201+0000 7f24d1000640 1 --2- 172.21.15.26:0/585237696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24cc162740 0x7f24cc155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:56.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.202+0000 7f24cb400640 1 --2- 172.21.15.26:0/585237696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24cc162740 0x7f24cc155ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:56.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.202+0000 7f24cb400640 1 -- 172.21.15.26:0/585237696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24cc15d1e0 con 0x7f24cc160930 2024-04-26T21:09:56.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.202+0000 7f24cb400640 1 --2- 172.21.15.26:0/585237696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24cc160930 0x7f24cc0c7f70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f24bc000c00 tx=0x7f24bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:56.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.202+0000 7f24c3e00640 1 -- 172.21.15.26:0/585237696 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc041020 con 0x7f24cc160930 2024-04-26T21:09:56.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.202+0000 7f24c3e00640 1 -- 172.21.15.26:0/585237696 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24bc030640 con 0x7f24cc160930 2024-04-26T21:09:56.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.202+0000 7f24c3e00640 1 -- 172.21.15.26:0/585237696 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc037040 con 0x7f24cc160930 2024-04-26T21:09:56.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.202+0000 7f24d1a00640 1 -- 172.21.15.26:0/585237696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24cc155ff0 con 0x7f24cc160930 2024-04-26T21:09:56.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.202+0000 7f24d1a00640 1 -- 172.21.15.26:0/585237696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24cc0c7000 con 0x7f24cc160930 2024-04-26T21:09:56.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.204+0000 7f24d1a00640 1 -- 172.21.15.26:0/585237696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24cc06a700 con 0x7f24cc160930 2024-04-26T21:09:56.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.204+0000 7f24c3e00640 1 -- 172.21.15.26:0/585237696 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24bc044020 con 0x7f24cc160930 2024-04-26T21:09:56.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.205+0000 7f24c3e00640 1 --2- 172.21.15.26:0/585237696 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f24a806c8f0 0x7f24a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:56.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.205+0000 7f24c3e00640 1 -- 172.21.15.26:0/585237696 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f24bc0ac3e0 con 0x7f24cc160930 2024-04-26T21:09:56.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.206+0000 7f24cbe00640 1 --2- 172.21.15.26:0/585237696 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f24a806c8f0 0x7f24a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:56.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.206+0000 7f24cbe00640 1 --2- 172.21.15.26:0/585237696 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f24a806c8f0 0x7f24a806edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009ad0 tx=0x7f24b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:56.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.210+0000 7f24c3e00640 1 -- 172.21.15.26:0/585237696 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f24bc04b030 con 0x7f24cc160930 2024-04-26T21:09:56.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.458+0000 7f24d1a00640 1 -- 172.21.15.26:0/585237696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f24cc0cd1b0 con 0x7f24cc160930 2024-04-26T21:09:56.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.458+0000 7f24c3e00640 1 -- 172.21.15.26:0/585237696 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f24bc07a780 con 0x7f24cc160930 2024-04-26T21:09:56.459 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:09:56.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.461+0000 7f24d1a00640 1 -- 172.21.15.26:0/585237696 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f24a806c8f0 msgr2=0x7f24a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:56.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.461+0000 7f24d1a00640 1 --2- 172.21.15.26:0/585237696 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f24a806c8f0 0x7f24a806edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009ad0 tx=0x7f24b4008040 comp rx=0 tx=0).stop 2024-04-26T21:09:56.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.461+0000 7f24d1a00640 1 -- 172.21.15.26:0/585237696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24cc160930 msgr2=0x7f24cc0c7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:56.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.461+0000 7f24d1a00640 1 --2- 172.21.15.26:0/585237696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24cc160930 0x7f24cc0c7f70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f24bc000c00 tx=0x7f24bc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:09:56.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.461+0000 7f24d1a00640 1 -- 172.21.15.26:0/585237696 shutdown_connections 2024-04-26T21:09:56.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.461+0000 7f24d1a00640 1 --2- 172.21.15.26:0/585237696 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f24a806c8f0 0x7f24a806edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:56.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.461+0000 7f24d1a00640 1 --2- 172.21.15.26:0/585237696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24cc162740 0x7f24cc155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:56.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.461+0000 7f24d1a00640 1 --2- 172.21.15.26:0/585237696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24cc160930 0x7f24cc0c7f70 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:56.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.461+0000 7f24d1a00640 1 --2- 172.21.15.26:0/585237696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24cc0b2e30 0x7f24cc0c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:56.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.461+0000 7f24d1a00640 1 -- 172.21.15.26:0/585237696 >> 172.21.15.26:0/585237696 conn(0x7f24cc06c660 msgr2=0x7f24cc1616c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:56.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.461+0000 7f24d1a00640 1 -- 172.21.15.26:0/585237696 shutdown_connections 2024-04-26T21:09:56.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56.461+0000 7f24d1a00640 1 -- 172.21.15.26:0/585237696 wait complete. 2024-04-26T21:09:56.482 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T21:09:56.482 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:09:56.482 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-04-26T21:09:56.482 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:09:56.482 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:09:56.482 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T21:09:56.482 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T21:09:56.482 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:09:56.482 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:09:56.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:56.483 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-04-26T21:09:56.536 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:09:56.536 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:09:56.536 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:09:56.536 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:56.536 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-04-26T21:09:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:56.537 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:09:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-04-26T21:09:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-04-26T21:09:56.537 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:09:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-04-26T21:09:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-04-26T21:09:56.537 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:09:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:09:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-04-26T21:09:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T21:09:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42588sec preferred_lft 42588sec 2024-04-26T21:09:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-04-26T21:09:56.537 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:56.537 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:09:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:09:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-04-26T21:09:56.538 INFO:teuthology.orchestra.run.smithi026.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T21:09:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 6a:d4:79:b9:a0:21 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:09:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T21:09:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::7072:11ff:fee4:1709/64 scope link 2024-04-26T21:09:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:56.538 INFO:teuthology.orchestra.run.smithi026.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:09:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 6a:d4:79:b9:a0:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:09:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::68d4:79ff:feb9:a021/64 scope link 2024-04-26T21:09:56.538 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:56.538 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T21:09:56.539 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether fe:99:1c:bd:47:17 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:09:56.539 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::fc99:1cff:febd:4717/64 scope link 2024-04-26T21:09:56.539 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:09:56.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:56.540 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:09:56.619 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-04-26T21:09:56.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:56.624 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-04-26T21:09:56.681 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-04-26T21:09:56.681 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:09:56.681 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:09:56.681 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:09:56.681 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:09:56.682 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:09:56.682 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-04-26T21:09:56.683 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:09:56.683 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:09:56.683 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:09:56.683 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-04-26T21:09:56.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:09:56.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:56.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:56.907 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-04-26T21:09:56.961 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:09:56.961 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:09:56.962 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:09:56.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:56.962 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-04-26T21:09:57.051 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:09:57.051 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:09:57.107 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:09:57.107 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:09:57.108 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:09:57.157 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-26T21:09:57.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:57.158 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:09:57.295 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:09:57.295 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:09:57.332 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-26T21:09:57.333 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-26T21:09:57.333 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.86:3300,172.21.15.86:3301" 2024-04-26T21:09:57.333 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with new device syntax: 0@eaf60df9-83a0-4283-ab2c-83d2c02a2067.cephfs=/ 2024-04-26T21:09:57.333 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.26:3300/172.21.15.86:3300/172.21.15.86:3301" will pass to kernel 2024-04-26T21:09:57.337 INFO:teuthology.orchestra.run.smithi026.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:09:57.337 INFO:teuthology.orchestra.run.smithi026.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:09:57.337 INFO:teuthology.orchestra.run.smithi026.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:09:57.337 INFO:teuthology.orchestra.run.smithi026.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:09:57.338 INFO:teuthology.orchestra.run.smithi026.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:09:57.342 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:09:57.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:57.343 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:09:57.425 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:09:57.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:09:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:57.570 INFO:teuthology.orchestra.run.smithi026.stdout:client4849 2024-04-26T21:09:57.571 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:09:57.571 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:09:57.626 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_symlink_recovery false 2024-04-26T21:09:57.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.835+0000 7f1fade00640 1 Processor -- start 2024-04-26T21:09:57.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.835+0000 7f1fade00640 1 -- start start 2024-04-26T21:09:57.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.837+0000 7f1fade00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1fa8160a80 0x7f1fa80b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:57.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.837+0000 7f1fade00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1fa8160e60 0x7f1fa806e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:57.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.837+0000 7f1fade00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fa8072d90 con 0x7f1fa8072490 2024-04-26T21:09:57.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.837+0000 7f1fade00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f1fa80b5be0 con 0x7f1fa8160e60 2024-04-26T21:09:57.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.837+0000 7f1fade00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1fa8073070 con 0x7f1fa8160a80 2024-04-26T21:09:57.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.837+0000 7f1fa7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1fa8160e60 0x7f1fa806e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:57.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.837+0000 7f1fa7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1fa8160e60 0x7f1fa806e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51228/0 (socket says 172.21.15.26:51228) 2024-04-26T21:09:57.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.838+0000 7f1fa7e00640 1 -- 172.21.15.26:0/3498212915 learned_addr learned my addr 172.21.15.26:0/3498212915 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:57.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.838+0000 7f1faca00640 1 --2- 172.21.15.26:0/3498212915 >> v2:172.21.15.86:3300/0 conn(0x7f1fa8160a80 0x7f1fa80b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:57.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.838+0000 7f1fa7e00640 1 -- 172.21.15.26:0/3498212915 >> v2:172.21.15.86:3300/0 conn(0x7f1fa8160a80 msgr2=0x7f1fa80b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:57.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.838+0000 7f1fa7e00640 1 --2- 172.21.15.26:0/3498212915 >> v2:172.21.15.86:3300/0 conn(0x7f1fa8160a80 0x7f1fa80b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:57.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.838+0000 7f1fa7e00640 1 -- 172.21.15.26:0/3498212915 >> v1:172.21.15.86:6789/0 conn(0x7f1fa8072490 legacy=0x7f1fa80b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:57.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.838+0000 7f1fa7e00640 1 -- 172.21.15.26:0/3498212915 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa8073350 con 0x7f1fa8160e60 2024-04-26T21:09:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.839+0000 7f1fa7e00640 1 --2- 172.21.15.26:0/3498212915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fa8160e60 0x7f1fa806e210 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8051090 tx=0x7f1f9402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc2fed8f8c80b10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.840+0000 7f1fa7400640 1 -- 172.21.15.26:0/3498212915 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94037070 con 0x7f1fa8160e60 2024-04-26T21:09:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.840+0000 7f1fa7400640 1 -- 172.21.15.26:0/3498212915 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f94032440 con 0x7f1fa8160e60 2024-04-26T21:09:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.840+0000 7f1fa7400640 1 -- 172.21.15.26:0/3498212915 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94032810 con 0x7f1fa8160e60 2024-04-26T21:09:57.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.840+0000 7f1fade00640 1 -- 172.21.15.26:0/3498212915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fa8160e60 msgr2=0x7f1fa806e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:57.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.840+0000 7f1fade00640 1 --2- 172.21.15.26:0/3498212915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fa8160e60 0x7f1fa806e210 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8051090 tx=0x7f1f9402a230 comp rx=0 tx=0).stop 2024-04-26T21:09:57.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.840+0000 7f1fade00640 1 -- 172.21.15.26:0/3498212915 shutdown_connections 2024-04-26T21:09:57.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.840+0000 7f1fade00640 1 --2- 172.21.15.26:0/3498212915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fa8160e60 0x7f1fa806e210 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:57.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.840+0000 7f1fade00640 1 --2- 172.21.15.26:0/3498212915 >> v2:172.21.15.86:3300/0 conn(0x7f1fa8160a80 0x7f1fa80b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:57.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.840+0000 7f1fade00640 1 -- 172.21.15.26:0/3498212915 >> 172.21.15.26:0/3498212915 conn(0x7f1fa80b2a10 msgr2=0x7f1fa80b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:57.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.840+0000 7f1fade00640 1 -- 172.21.15.26:0/3498212915 shutdown_connections 2024-04-26T21:09:57.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.840+0000 7f1fade00640 1 -- 172.21.15.26:0/3498212915 wait complete. 2024-04-26T21:09:57.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.842+0000 7f1fade00640 1 Processor -- start 2024-04-26T21:09:57.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.842+0000 7f1fade00640 1 -- start start 2024-04-26T21:09:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.843+0000 7f1fade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa8072490 0x7f1fa8144fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.843+0000 7f1fade00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fa8160a80 0x7f1fa814a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.843+0000 7f1fade00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fa8160e60 0x7f1fa814a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.843+0000 7f1fade00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa81612d0 con 0x7f1fa8160e60 2024-04-26T21:09:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.843+0000 7f1fade00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa80b5820 con 0x7f1fa8072490 2024-04-26T21:09:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.843+0000 7f1fade00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa8072900 con 0x7f1fa8160a80 2024-04-26T21:09:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.843+0000 7f1fad400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fa8160e60 0x7f1fa814a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.844+0000 7f1fad400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fa8160e60 0x7f1fa814a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51242/0 (socket says 172.21.15.26:51242) 2024-04-26T21:09:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.844+0000 7f1fad400640 1 -- 172.21.15.26:0/2602255083 learned_addr learned my addr 172.21.15.26:0/2602255083 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:57.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.844+0000 7f1fad400640 1 -- 172.21.15.26:0/2602255083 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fa8160a80 msgr2=0x7f1fa814a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:57.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.844+0000 7f1fad400640 1 --2- 172.21.15.26:0/2602255083 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fa8160a80 0x7f1fa814a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:57.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.844+0000 7f1fad400640 1 -- 172.21.15.26:0/2602255083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa8072490 msgr2=0x7f1fa8144fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:57.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.844+0000 7f1fad400640 1 --2- 172.21.15.26:0/2602255083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa8072490 0x7f1fa8144fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:57.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.844+0000 7f1fad400640 1 -- 172.21.15.26:0/2602255083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa815cac0 con 0x7f1fa8160e60 2024-04-26T21:09:57.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.844+0000 7f1fad400640 1 --2- 172.21.15.26:0/2602255083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fa8160e60 0x7f1fa814a890 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000c780 tx=0x7f1fa000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:57.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.844+0000 7f1fa4c00640 1 -- 172.21.15.26:0/2602255083 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0017070 con 0x7f1fa8160e60 2024-04-26T21:09:57.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.845+0000 7f1fa4c00640 1 -- 172.21.15.26:0/2602255083 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fa0003da0 con 0x7f1fa8160e60 2024-04-26T21:09:57.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.845+0000 7f1fa4c00640 1 -- 172.21.15.26:0/2602255083 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa00052b0 con 0x7f1fa8160e60 2024-04-26T21:09:57.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.845+0000 7f1fade00640 1 -- 172.21.15.26:0/2602255083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa815cda0 con 0x7f1fa8160e60 2024-04-26T21:09:57.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.845+0000 7f1fade00640 1 -- 172.21.15.26:0/2602255083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa80b0d00 con 0x7f1fa8160e60 2024-04-26T21:09:57.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.846+0000 7f1fade00640 1 -- 172.21.15.26:0/2602255083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa806a700 con 0x7f1fa8160e60 2024-04-26T21:09:57.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.847+0000 7f1fa4c00640 1 -- 172.21.15.26:0/2602255083 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1fa0005450 con 0x7f1fa8160e60 2024-04-26T21:09:57.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.848+0000 7f1fa4c00640 1 --2- 172.21.15.26:0/2602255083 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1f8806c8f0 0x7f1f8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:57.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.848+0000 7f1fa4c00640 1 -- 172.21.15.26:0/2602255083 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f1fa008b130 con 0x7f1fa8160e60 2024-04-26T21:09:57.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.849+0000 7f1faca00640 1 --2- 172.21.15.26:0/2602255083 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1f8806c8f0 0x7f1f8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:57.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.849+0000 7f1faca00640 1 --2- 172.21.15.26:0/2602255083 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1f8806c8f0 0x7f1f8806edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0098f0 tx=0x7f1f9c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:57.853+0000 7f1fa4c00640 1 -- 172.21.15.26:0/2602255083 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1fa0059550 con 0x7f1fa8160e60 2024-04-26T21:09:58.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.060+0000 7f1fade00640 1 -- 172.21.15.26:0/2602255083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mds_symlink_recovery}] v 0) -- 0x7f1fa8072270 con 0x7f1fa8160e60 2024-04-26T21:09:58.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.068+0000 7f1fa4c00640 1 -- 172.21.15.26:0/2602255083 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_symlink_recovery}]=0 v11)=0 v11) ==== 122+0+0 (secure 0 0 0) 0x7f1fa0058e50 con 0x7f1fa8160e60 2024-04-26T21:09:58.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.071+0000 7f1fade00640 1 -- 172.21.15.26:0/2602255083 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1f8806c8f0 msgr2=0x7f1f8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:58.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.071+0000 7f1fade00640 1 --2- 172.21.15.26:0/2602255083 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1f8806c8f0 0x7f1f8806edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0098f0 tx=0x7f1f9c008040 comp rx=0 tx=0).stop 2024-04-26T21:09:58.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.071+0000 7f1fade00640 1 -- 172.21.15.26:0/2602255083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fa8160e60 msgr2=0x7f1fa814a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:58.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.071+0000 7f1fade00640 1 --2- 172.21.15.26:0/2602255083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fa8160e60 0x7f1fa814a890 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000c780 tx=0x7f1fa000cc40 comp rx=0 tx=0).stop 2024-04-26T21:09:58.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.071+0000 7f1fade00640 1 -- 172.21.15.26:0/2602255083 shutdown_connections 2024-04-26T21:09:58.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.071+0000 7f1fade00640 1 --2- 172.21.15.26:0/2602255083 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1f8806c8f0 0x7f1f8806edb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:58.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.071+0000 7f1fade00640 1 --2- 172.21.15.26:0/2602255083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fa8160e60 0x7f1fa814a890 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:58.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.071+0000 7f1fade00640 1 --2- 172.21.15.26:0/2602255083 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fa8160a80 0x7f1fa814a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:58.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.071+0000 7f1fade00640 1 --2- 172.21.15.26:0/2602255083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fa8072490 0x7f1fa8144fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:58.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.071+0000 7f1fade00640 1 -- 172.21.15.26:0/2602255083 >> 172.21.15.26:0/2602255083 conn(0x7f1fa80b2a10 msgr2=0x7f1fa815e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:58.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.071+0000 7f1fade00640 1 -- 172.21.15.26:0/2602255083 shutdown_connections 2024-04-26T21:09:58.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.071+0000 7f1fade00640 1 -- 172.21.15.26:0/2602255083 wait complete. 2024-04-26T21:09:58.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:58.092 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir sym_dir0) 2024-04-26T21:09:58.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:58.159 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch sym_dir0/file1) 2024-04-26T21:09:58.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:58.215 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln -s sym_dir0/file1 sym_dir0/symlink_file1) 2024-04-26T21:09:58.271 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-26T21:09:58.271 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-26T21:09:58.271 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-26T21:09:58.271 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-26T21:09:58.271 DEBUG:teuthology.orchestra.run.smithi026:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/sym_dir0/symlink_file1").st_ino) 2024-04-26T21:09:58.271 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-26T21:09:58.425 INFO:teuthology.orchestra.run.smithi026.stdout:1099511628278 2024-04-26T21:09:58.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:09:58.440 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:09:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.698+0000 7f5ec1a00640 1 Processor -- start 2024-04-26T21:09:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.698+0000 7f5ec1a00640 1 -- start start 2024-04-26T21:09:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.699+0000 7f5ec1a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5ebc162ba0 0x7f5ebc0d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.699+0000 7f5ec1a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5ebc0d2060 0x7f5ebc0d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.699+0000 7f5ec1a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ebc0b61c0 con 0x7f5ebc0d1c80 2024-04-26T21:09:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.699+0000 7f5ec1a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f5ebc0b5f70 con 0x7f5ebc162ba0 2024-04-26T21:09:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.699+0000 7f5ec1a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5ebc0d3830 con 0x7f5ebc0d2060 2024-04-26T21:09:58.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.699+0000 7f5eb3e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5ebc162ba0 0x7f5ebc0d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:58.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.699+0000 7f5ebbe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5ebc0d1c80 0x7f5ebc162490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50196/0 (socket says 172.21.15.26:50196) 2024-04-26T21:09:58.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.699+0000 7f5ebbe00640 1 -- 172.21.15.26:0/3671002581 learned_addr learned my addr 172.21.15.26:0/3671002581 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:09:58.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.700+0000 7f5ec1000640 1 --2- 172.21.15.26:0/3671002581 >> v2:172.21.15.86:3300/0 conn(0x7f5ebc0d2060 0x7f5ebc0d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:58.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.700+0000 7f5eb3e00640 1 -- 172.21.15.26:0/3671002581 >> v2:172.21.15.86:3300/0 conn(0x7f5ebc0d2060 msgr2=0x7f5ebc0d3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:58.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.700+0000 7f5eb3e00640 1 --2- 172.21.15.26:0/3671002581 >> v2:172.21.15.86:3300/0 conn(0x7f5ebc0d2060 0x7f5ebc0d3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:58.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.700+0000 7f5eb3e00640 1 -- 172.21.15.26:0/3671002581 >> v1:172.21.15.26:6789/0 conn(0x7f5ebc0d1c80 legacy=0x7f5ebc162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:58.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.700+0000 7f5eb3e00640 1 -- 172.21.15.26:0/3671002581 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc0b64a0 con 0x7f5ebc162ba0 2024-04-26T21:09:58.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.700+0000 7f5ebb400640 1 -- 172.21.15.26:0/3671002581 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 908181752 0 0) 0x7f5ebc0b61c0 con 0x7f5ebc0d1c80 2024-04-26T21:09:58.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.701+0000 7f5eb3e00640 1 --2- 172.21.15.26:0/3671002581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ebc162ba0 0x7f5ebc0d0cf0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0d17b0 tx=0x7f5ea402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4bec62c5ec62be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.702+0000 7f5ebb400640 1 -- 172.21.15.26:0/3671002581 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4037070 con 0x7f5ebc162ba0 2024-04-26T21:09:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.702+0000 7f5ebb400640 1 -- 172.21.15.26:0/3671002581 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea4032440 con 0x7f5ebc162ba0 2024-04-26T21:09:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.702+0000 7f5ebb400640 1 -- 172.21.15.26:0/3671002581 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea402ae60 con 0x7f5ebc162ba0 2024-04-26T21:09:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.702+0000 7f5ec1a00640 1 -- 172.21.15.26:0/3671002581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ebc162ba0 msgr2=0x7f5ebc0d0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.702+0000 7f5ec1a00640 1 --2- 172.21.15.26:0/3671002581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ebc162ba0 0x7f5ebc0d0cf0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0d17b0 tx=0x7f5ea402a110 comp rx=0 tx=0).stop 2024-04-26T21:09:58.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.702+0000 7f5ec1a00640 1 -- 172.21.15.26:0/3671002581 shutdown_connections 2024-04-26T21:09:58.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.702+0000 7f5ec1a00640 1 --2- 172.21.15.26:0/3671002581 >> v2:172.21.15.86:3300/0 conn(0x7f5ebc0d2060 0x7f5ebc0d3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:58.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.702+0000 7f5ec1a00640 1 --2- 172.21.15.26:0/3671002581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ebc162ba0 0x7f5ebc0d0cf0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:58.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.702+0000 7f5ec1a00640 1 -- 172.21.15.26:0/3671002581 >> 172.21.15.26:0/3671002581 conn(0x7f5ebc0d5020 msgr2=0x7f5ebc0d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:58.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.703+0000 7f5ec1a00640 1 -- 172.21.15.26:0/3671002581 shutdown_connections 2024-04-26T21:09:58.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.703+0000 7f5ec1a00640 1 -- 172.21.15.26:0/3671002581 wait complete. 2024-04-26T21:09:58.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.705+0000 7f5ec1a00640 1 Processor -- start 2024-04-26T21:09:58.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.705+0000 7f5ec1a00640 1 -- start start 2024-04-26T21:09:58.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.706+0000 7f5ec1a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ebc0d1c80 0x7f5ebc14f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:58.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.706+0000 7f5ec1a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ebc0d2060 0x7f5ebc0ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:58.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.706+0000 7f5ec1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ebc162ba0 0x7f5ebc15de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:58.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.706+0000 7f5ec1a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc0d5cc0 con 0x7f5ebc0d2060 2024-04-26T21:09:58.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.706+0000 7f5ec1a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc0d5b40 con 0x7f5ebc162ba0 2024-04-26T21:09:58.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.706+0000 7f5ec1a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5ebc0c8b70 con 0x7f5ebc0d1c80 2024-04-26T21:09:58.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.707+0000 7f5eb3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ebc0d2060 0x7f5ebc0ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:58.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.707+0000 7f5eb3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ebc0d2060 0x7f5ebc0ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51268/0 (socket says 172.21.15.26:51268) 2024-04-26T21:09:58.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.707+0000 7f5eb3e00640 1 -- 172.21.15.26:0/3663749502 learned_addr learned my addr 172.21.15.26:0/3663749502 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:58.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.707+0000 7f5ebbe00640 1 --2- 172.21.15.26:0/3663749502 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ebc0d1c80 0x7f5ebc14f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:58.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.707+0000 7f5ec1000640 1 --2- 172.21.15.26:0/3663749502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ebc162ba0 0x7f5ebc15de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:58.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.707+0000 7f5eb3e00640 1 -- 172.21.15.26:0/3663749502 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ebc0d1c80 msgr2=0x7f5ebc14f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:58.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.707+0000 7f5eb3e00640 1 --2- 172.21.15.26:0/3663749502 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ebc0d1c80 0x7f5ebc14f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:58.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.707+0000 7f5eb3e00640 1 -- 172.21.15.26:0/3663749502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ebc162ba0 msgr2=0x7f5ebc15de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:58.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.707+0000 7f5eb3e00640 1 --2- 172.21.15.26:0/3663749502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ebc162ba0 0x7f5ebc15de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:58.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.707+0000 7f5eb3e00640 1 -- 172.21.15.26:0/3663749502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc14fa30 con 0x7f5ebc0d2060 2024-04-26T21:09:58.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.708+0000 7f5eb3e00640 1 --2- 172.21.15.26:0/3663749502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ebc0d2060 0x7f5ebc0ccb00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009890 tx=0x7f5ea40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:58.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.708+0000 7f5eb8c00640 1 -- 172.21.15.26:0/3663749502 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4037040 con 0x7f5ebc0d2060 2024-04-26T21:09:58.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.708+0000 7f5eb8c00640 1 -- 172.21.15.26:0/3663749502 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea4042070 con 0x7f5ebc0d2060 2024-04-26T21:09:58.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.708+0000 7f5ec1a00640 1 -- 172.21.15.26:0/3663749502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ebc159de0 con 0x7f5ebc0d2060 2024-04-26T21:09:58.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.708+0000 7f5eb8c00640 1 -- 172.21.15.26:0/3663749502 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4030bd0 con 0x7f5ebc0d2060 2024-04-26T21:09:58.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.708+0000 7f5ec1a00640 1 -- 172.21.15.26:0/3663749502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ebc15a3a0 con 0x7f5ebc0d2060 2024-04-26T21:09:58.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.710+0000 7f5ec1a00640 1 -- 172.21.15.26:0/3663749502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ebc06a700 con 0x7f5ebc0d2060 2024-04-26T21:09:58.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.711+0000 7f5eb8c00640 1 -- 172.21.15.26:0/3663749502 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ea4047460 con 0x7f5ebc0d2060 2024-04-26T21:09:58.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.712+0000 7f5eb8c00640 1 --2- 172.21.15.26:0/3663749502 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5e9806c8f0 0x7f5e9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:58.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.712+0000 7f5eb8c00640 1 -- 172.21.15.26:0/3663749502 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f5ea40ac740 con 0x7f5ebc0d2060 2024-04-26T21:09:58.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.712+0000 7f5ebbe00640 1 --2- 172.21.15.26:0/3663749502 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5e9806c8f0 0x7f5e9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:58.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.713+0000 7f5ebbe00640 1 --2- 172.21.15.26:0/3663749502 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5e9806c8f0 0x7f5e9806edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5eac000f60 tx=0x7f5eac007b50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:58.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.717+0000 7f5eb8c00640 1 -- 172.21.15.26:0/3663749502 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5ea4032e20 con 0x7f5ebc0d2060 2024-04-26T21:09:58.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.961+0000 7f5ec1a00640 1 -- 172.21.15.26:0/3663749502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ebc161ca0 con 0x7f5ebc0d2060 2024-04-26T21:09:58.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.962+0000 7f5eb8c00640 1 -- 172.21.15.26:0/3663749502 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7f5ea407ab60 con 0x7f5ebc0d2060 2024-04-26T21:09:58.964 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:09:58.964 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2349160700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2349160700},{"type":"v1","addr":"172.21.15.26:6837","nonce":2349160700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4598,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2824191762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2824191762},{"type":"v1","addr":"172.21.15.86:6834","nonce":2824191762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/263746310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":263746310},{"type":"v1","addr":"172.21.15.86:6835","nonce":263746310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:09:48.622025+0000","modified":"2024-04-26T21:09:51.721103+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5058":{"gid":5058,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6836/2528180854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2528180854},{"type":"v1","addr":"172.21.15.26:6836","nonce":2528180854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5058,"qdb_cluster":[5058]},"id":2}]} 2024-04-26T21:09:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 17 2024-04-26T21:09:58.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.965+0000 7f5ec1a00640 1 -- 172.21.15.26:0/3663749502 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5e9806c8f0 msgr2=0x7f5e9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:58.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.965+0000 7f5ec1a00640 1 --2- 172.21.15.26:0/3663749502 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5e9806c8f0 0x7f5e9806edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5eac000f60 tx=0x7f5eac007b50 comp rx=0 tx=0).stop 2024-04-26T21:09:58.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.965+0000 7f5ec1a00640 1 -- 172.21.15.26:0/3663749502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ebc0d2060 msgr2=0x7f5ebc0ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:58.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.965+0000 7f5ec1a00640 1 --2- 172.21.15.26:0/3663749502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ebc0d2060 0x7f5ebc0ccb00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009890 tx=0x7f5ea40046c0 comp rx=0 tx=0).stop 2024-04-26T21:09:58.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.965+0000 7f5ec1a00640 1 -- 172.21.15.26:0/3663749502 shutdown_connections 2024-04-26T21:09:58.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.965+0000 7f5ec1a00640 1 --2- 172.21.15.26:0/3663749502 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5e9806c8f0 0x7f5e9806edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:58.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.965+0000 7f5ec1a00640 1 --2- 172.21.15.26:0/3663749502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ebc162ba0 0x7f5ebc15de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:58.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.965+0000 7f5ec1a00640 1 --2- 172.21.15.26:0/3663749502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ebc0d2060 0x7f5ebc0ccb00 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:58.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.965+0000 7f5ec1a00640 1 --2- 172.21.15.26:0/3663749502 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ebc0d1c80 0x7f5ebc14f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:58.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.965+0000 7f5ec1a00640 1 -- 172.21.15.26:0/3663749502 >> 172.21.15.26:0/3663749502 conn(0x7f5ebc0d5020 msgr2=0x7f5ebc0d2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:58.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.966+0000 7f5ec1a00640 1 -- 172.21.15.26:0/3663749502 shutdown_connections 2024-04-26T21:09:58.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:58.966+0000 7f5ec1a00640 1 -- 172.21.15.26:0/3663749502 wait complete. 2024-04-26T21:09:58.986 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-26T21:09:59.227 INFO:teuthology.orchestra.run.smithi026.stdout:{"message":"","return_code":0} 2024-04-26T21:09:59.228 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-26T21:09:59.228 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f6.00000000 symlink 2024-04-26T21:09:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.317+0000 7f0feb861f40 1 Processor -- start 2024-04-26T21:09:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.318+0000 7f0feb861f40 1 -- start start 2024-04-26T21:09:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.318+0000 7f0feb861f40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559fd49e47a0 0x559fd499a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.318+0000 7f0feb861f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fd4a704b0 0x559fd4a67050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.318+0000 7f0feb861f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fd4a77660 0x559fd4a6e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.318+0000 7f0feb861f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559fd4941d20 con 0x559fd4a77660 2024-04-26T21:09:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.318+0000 7f0feb861f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559fd4945d60 con 0x559fd4a704b0 2024-04-26T21:09:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.318+0000 7f0feb861f40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x559fd49e3c30 con 0x559fd49e47a0 2024-04-26T21:09:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.319+0000 7f0fe9400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fd4a77660 0x559fd4a6e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.319+0000 7f0fe9400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fd4a77660 0x559fd4a6e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47326/0 (socket says 172.21.15.26:47326) 2024-04-26T21:09:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.319+0000 7f0fe9400640 1 -- 172.21.15.26:0/92237375 learned_addr learned my addr 172.21.15.26:0/92237375 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.319+0000 7f0fe8a00640 1 --2- 172.21.15.26:0/92237375 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559fd49e47a0 0x559fd499a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.319+0000 7f0fe8000640 1 --2- 172.21.15.26:0/92237375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fd4a704b0 0x559fd4a67050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.319+0000 7f0fe9400640 1 -- 172.21.15.26:0/92237375 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559fd49e47a0 msgr2=0x559fd499a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.319+0000 7f0fe9400640 1 --2- 172.21.15.26:0/92237375 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559fd49e47a0 0x559fd499a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.319+0000 7f0fe9400640 1 -- 172.21.15.26:0/92237375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fd4a704b0 msgr2=0x559fd4a67050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.319+0000 7f0fe9400640 1 --2- 172.21.15.26:0/92237375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fd4a704b0 0x559fd4a67050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.319+0000 7f0fe9400640 1 -- 172.21.15.26:0/92237375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fd4a791a0 con 0x559fd4a77660 2024-04-26T21:09:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.320+0000 7f0fe9400640 1 --2- 172.21.15.26:0/92237375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fd4a77660 0x559fd4a6e160 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000a3e0 tx=0x7f0fe000a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890f226bdcff8f5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.320+0000 7f0fe6c00640 1 -- 172.21.15.26:0/92237375 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe000b920 con 0x559fd4a77660 2024-04-26T21:09:59.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.320+0000 7f0fe6c00640 1 -- 172.21.15.26:0/92237375 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fe000f030 con 0x559fd4a77660 2024-04-26T21:09:59.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.320+0000 7f0fe6c00640 1 -- 172.21.15.26:0/92237375 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe000bd20 con 0x559fd4a77660 2024-04-26T21:09:59.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.320+0000 7f0feb861f40 1 -- 172.21.15.26:0/92237375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fd4a77660 msgr2=0x559fd4a6e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:59.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.320+0000 7f0feb861f40 1 --2- 172.21.15.26:0/92237375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fd4a77660 0x559fd4a6e160 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000a3e0 tx=0x7f0fe000a8a0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.320+0000 7f0feb861f40 1 -- 172.21.15.26:0/92237375 shutdown_connections 2024-04-26T21:09:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.320+0000 7f0feb861f40 1 --2- 172.21.15.26:0/92237375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fd4a77660 0x559fd4a6e160 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.320+0000 7f0feb861f40 1 --2- 172.21.15.26:0/92237375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fd4a704b0 0x559fd4a67050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.320+0000 7f0feb861f40 1 --2- 172.21.15.26:0/92237375 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559fd49e47a0 0x559fd499a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.320+0000 7f0feb861f40 1 -- 172.21.15.26:0/92237375 >> 172.21.15.26:0/92237375 conn(0x559fd49cb960 msgr2=0x559fd49c67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.321+0000 7f0feb861f40 1 -- 172.21.15.26:0/92237375 shutdown_connections 2024-04-26T21:09:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.321+0000 7f0feb861f40 1 -- 172.21.15.26:0/92237375 wait complete. 2024-04-26T21:09:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.321+0000 7f0feb861f40 1 Processor -- start 2024-04-26T21:09:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.321+0000 7f0feb861f40 1 -- start start 2024-04-26T21:09:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.322+0000 7f0feb861f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fd49e47a0 0x559fd4a5dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.322+0000 7f0feb861f40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559fd4a704b0 0x559fd499a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.322+0000 7f0feb861f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fd4a77660 0x559fd4a70890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.322+0000 7f0feb861f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559fd49c7070 con 0x559fd4a77660 2024-04-26T21:09:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.322+0000 7f0feb861f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559fd4a79410 con 0x559fd49e47a0 2024-04-26T21:09:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.322+0000 7f0feb861f40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x559fd49c7350 con 0x559fd4a704b0 2024-04-26T21:09:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.323+0000 7f0fe9400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fd4a77660 0x559fd4a70890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.323+0000 7f0fe9400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fd4a77660 0x559fd4a70890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47340/0 (socket says 172.21.15.26:47340) 2024-04-26T21:09:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.323+0000 7f0fe9400640 1 -- 172.21.15.26:0/1358132914 learned_addr learned my addr 172.21.15.26:0/1358132914 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.323+0000 7f0fe8a00640 1 --2- 172.21.15.26:0/1358132914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fd49e47a0 0x559fd4a5dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.323+0000 7f0fe8000640 1 --2- 172.21.15.26:0/1358132914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559fd4a704b0 0x559fd499a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.323+0000 7f0fe9400640 1 -- 172.21.15.26:0/1358132914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559fd4a704b0 msgr2=0x559fd499a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.323+0000 7f0fe9400640 1 --2- 172.21.15.26:0/1358132914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559fd4a704b0 0x559fd499a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.323+0000 7f0fe9400640 1 -- 172.21.15.26:0/1358132914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fd49e47a0 msgr2=0x559fd4a5dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.323+0000 7f0fe9400640 1 --2- 172.21.15.26:0/1358132914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fd49e47a0 0x559fd4a5dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.323+0000 7f0fe9400640 1 -- 172.21.15.26:0/1358132914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fd499ad10 con 0x559fd4a77660 2024-04-26T21:09:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.323+0000 7f0fe8a00640 1 --2- 172.21.15.26:0/1358132914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559fd49e47a0 0x559fd4a5dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:59.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.323+0000 7f0fe9400640 1 --2- 172.21.15.26:0/1358132914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fd4a77660 0x559fd4a70890 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000ada0 tx=0x7f0fe0002f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.324+0000 7f0fe5800640 1 -- 172.21.15.26:0/1358132914 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0024070 con 0x559fd4a77660 2024-04-26T21:09:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.324+0000 7f0fe5800640 1 -- 172.21.15.26:0/1358132914 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fe000cd10 con 0x559fd4a77660 2024-04-26T21:09:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.324+0000 7f0fe5800640 1 -- 172.21.15.26:0/1358132914 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0014660 con 0x559fd4a77660 2024-04-26T21:09:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.324+0000 7f0feb861f40 1 -- 172.21.15.26:0/1358132914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559fd499c5d0 con 0x559fd4a77660 2024-04-26T21:09:59.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.324+0000 7f0feb861f40 1 -- 172.21.15.26:0/1358132914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fd499cb90 con 0x559fd4a77660 2024-04-26T21:09:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.326+0000 7f0fe5800640 1 -- 172.21.15.26:0/1358132914 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0fe0027070 con 0x559fd4a77660 2024-04-26T21:09:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.327+0000 7f0fe5800640 1 --2- 172.21.15.26:0/1358132914 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0fc806b2d0 0x7f0fc806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.327+0000 7f0fe5800640 1 -- 172.21.15.26:0/1358132914 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f0fe008b570 con 0x559fd4a77660 2024-04-26T21:09:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.327+0000 7f0fe8a00640 1 --2- 172.21.15.26:0/1358132914 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0fc806b2d0 0x7f0fc806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.327+0000 7f0feb861f40 1 --2- 172.21.15.26:0/1358132914 >> [v2:172.21.15.86:6816/1450967413,v1:172.21.15.86:6817/1450967413] conn(0x559fd4a62de0 0x559fd4a63240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.327+0000 7f0feb861f40 1 -- 172.21.15.26:0/1358132914 --> [v2:172.21.15.86:6816/1450967413,v1:172.21.15.86:6817/1450967413] -- osd_op(unknown.0.0:1 5.2a 5:57fe7535:::100000001f6.00000000:head [getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) -- 0x559fd4b18ec0 con 0x559fd4a62de0 2024-04-26T21:09:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.328+0000 7f0fe8a00640 1 --2- 172.21.15.26:0/1358132914 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0fc806b2d0 0x7f0fc806d790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8004170 tx=0x7f0fd8000bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:59.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.328+0000 7f0fe8000640 1 --2- 172.21.15.26:0/1358132914 >> [v2:172.21.15.86:6816/1450967413,v1:172.21.15.86:6817/1450967413] conn(0x559fd4a62de0 0x559fd4a63240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:59.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.329+0000 7f0fe8000640 1 --2- 172.21.15.26:0/1358132914 >> [v2:172.21.15.86:6816/1450967413,v1:172.21.15.86:6817/1450967413] conn(0x559fd4a62de0 0x559fd4a63240 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:59.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.330+0000 7f0fe8000640 1 -- 172.21.15.26:0/1358132914 <== osd.4 v2:172.21.15.86:6816/1450967413 1 ==== osd_op_reply(1 100000001f6.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f0fd40527e0 con 0x559fd4a62de0 2024-04-26T21:09:59.332 INFO:teuthology.orchestra.run.smithi026.stderr:error getting xattr cephfs_data/100000001f6.00000000/symlink: (61) No data available 2024-04-26T21:09:59.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.330+0000 7f0feb861f40 1 -- 172.21.15.26:0/1358132914 >> [v2:172.21.15.86:6816/1450967413,v1:172.21.15.86:6817/1450967413] conn(0x559fd4a62de0 msgr2=0x559fd4a63240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:59.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.330+0000 7f0feb861f40 1 --2- 172.21.15.26:0/1358132914 >> [v2:172.21.15.86:6816/1450967413,v1:172.21.15.86:6817/1450967413] conn(0x559fd4a62de0 0x559fd4a63240 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.330+0000 7f0feb861f40 1 -- 172.21.15.26:0/1358132914 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0fc806b2d0 msgr2=0x7f0fc806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:59.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.330+0000 7f0feb861f40 1 --2- 172.21.15.26:0/1358132914 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0fc806b2d0 0x7f0fc806d790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8004170 tx=0x7f0fd8000bc0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.330+0000 7f0feb861f40 1 -- 172.21.15.26:0/1358132914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fd4a77660 msgr2=0x559fd4a70890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:59.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.330+0000 7f0feb861f40 1 --2- 172.21.15.26:0/1358132914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559fd4a77660 0x559fd4a70890 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000ada0 tx=0x7f0fe0002f70 comp rx=0 tx=0).stop 2024-04-26T21:09:59.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.330+0000 7f0fe9400640 1 -- 172.21.15.26:0/1358132914 reap_dead start 2024-04-26T21:09:59.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.330+0000 7f0feb861f40 1 -- 172.21.15.26:0/1358132914 shutdown_connections 2024-04-26T21:09:59.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.330+0000 7f0feb861f40 1 -- 172.21.15.26:0/1358132914 >> 172.21.15.26:0/1358132914 conn(0x559fd49cb960 msgr2=0x559fd49c7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:59.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.330+0000 7f0feb861f40 1 -- 172.21.15.26:0/1358132914 shutdown_connections 2024-04-26T21:09:59.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.331+0000 7f0feb861f40 1 -- 172.21.15.26:0/1358132914 wait complete. 2024-04-26T21:09:59.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:09:59.337 ERROR:tasks.cephfs.filesystem:Command failed on smithi026 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f6.00000000 symlink' 2024-04-26T21:09:59.337 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_symlink_recovery true 2024-04-26T21:09:59.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.532+0000 7fda85a00640 1 Processor -- start 2024-04-26T21:09:59.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.532+0000 7fda85a00640 1 -- start start 2024-04-26T21:09:59.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.533+0000 7fda85a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fda8015bdc0 0x7fda800b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:59.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.533+0000 7fda85a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fda80073530 0x7fda800cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:59.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.534+0000 7fda85a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda8006fbb0 con 0x7fda8006e210 2024-04-26T21:09:59.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.534+0000 7fda85a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fda800b4cf0 con 0x7fda8015bdc0 2024-04-26T21:09:59.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.534+0000 7fda85a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fda800cf280 con 0x7fda80073530 2024-04-26T21:09:59.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.534+0000 7fda7fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fda8015bdc0 0x7fda800b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:59.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.534+0000 7fda7fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fda8015bdc0 0x7fda800b47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47356/0 (socket says 172.21.15.26:47356) 2024-04-26T21:09:59.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.534+0000 7fda7fe00640 1 -- 172.21.15.26:0/1211290449 learned_addr learned my addr 172.21.15.26:0/1211290449 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:59.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.534+0000 7fda85000640 1 --2- 172.21.15.26:0/1211290449 >> v2:172.21.15.86:3300/0 conn(0x7fda80073530 0x7fda800cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:59.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.534+0000 7fda7fe00640 1 -- 172.21.15.26:0/1211290449 >> v2:172.21.15.86:3300/0 conn(0x7fda80073530 msgr2=0x7fda800cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:59.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.534+0000 7fda7fe00640 1 --2- 172.21.15.26:0/1211290449 >> v2:172.21.15.86:3300/0 conn(0x7fda80073530 0x7fda800cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.534+0000 7fda7fe00640 1 -- 172.21.15.26:0/1211290449 >> v1:172.21.15.86:6789/0 conn(0x7fda8006e210 legacy=0x7fda80072e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:09:59.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.534+0000 7fda7fe00640 1 -- 172.21.15.26:0/1211290449 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8006e780 con 0x7fda8015bdc0 2024-04-26T21:09:59.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.534+0000 7fda85000640 1 --2- 172.21.15.26:0/1211290449 >> v2:172.21.15.86:3300/0 conn(0x7fda80073530 0x7fda800cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:09:59.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.536+0000 7fda7fe00640 1 --2- 172.21.15.26:0/1211290449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda8015bdc0 0x7fda800b47b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fda800cf400 tx=0x7fda7002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd7e11c8c9a003c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:59.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.536+0000 7fda7f400640 1 -- 172.21.15.26:0/1211290449 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70037070 con 0x7fda8015bdc0 2024-04-26T21:09:59.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.536+0000 7fda7f400640 1 -- 172.21.15.26:0/1211290449 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda70032440 con 0x7fda8015bdc0 2024-04-26T21:09:59.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.536+0000 7fda7f400640 1 -- 172.21.15.26:0/1211290449 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70032810 con 0x7fda8015bdc0 2024-04-26T21:09:59.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.536+0000 7fda85a00640 1 -- 172.21.15.26:0/1211290449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda8015bdc0 msgr2=0x7fda800b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:59.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.536+0000 7fda85a00640 1 --2- 172.21.15.26:0/1211290449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda8015bdc0 0x7fda800b47b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fda800cf400 tx=0x7fda7002a180 comp rx=0 tx=0).stop 2024-04-26T21:09:59.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.537+0000 7fda85a00640 1 -- 172.21.15.26:0/1211290449 shutdown_connections 2024-04-26T21:09:59.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.537+0000 7fda85a00640 1 --2- 172.21.15.26:0/1211290449 >> v2:172.21.15.86:3300/0 conn(0x7fda80073530 0x7fda800cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.537+0000 7fda85a00640 1 --2- 172.21.15.26:0/1211290449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda8015bdc0 0x7fda800b47b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.537+0000 7fda85a00640 1 -- 172.21.15.26:0/1211290449 >> 172.21.15.26:0/1211290449 conn(0x7fda800b2a10 msgr2=0x7fda800b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:59.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.537+0000 7fda85a00640 1 -- 172.21.15.26:0/1211290449 shutdown_connections 2024-04-26T21:09:59.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.537+0000 7fda85a00640 1 -- 172.21.15.26:0/1211290449 wait complete. 2024-04-26T21:09:59.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.539+0000 7fda85a00640 1 Processor -- start 2024-04-26T21:09:59.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.539+0000 7fda85a00640 1 -- start start 2024-04-26T21:09:59.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.540+0000 7fda85a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda8006e210 0x7fda8013fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:59.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.540+0000 7fda85a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda80073530 0x7fda8014c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:59.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.540+0000 7fda85a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda8015bdc0 0x7fda8014cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:59.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.540+0000 7fda85a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fda80072380 con 0x7fda8015bdc0 2024-04-26T21:09:59.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.540+0000 7fda85a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fda80070130 con 0x7fda8006e210 2024-04-26T21:09:59.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.540+0000 7fda85a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fda8014d360 con 0x7fda80073530 2024-04-26T21:09:59.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.540+0000 7fda85000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda8015bdc0 0x7fda8014cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:59.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.540+0000 7fda85000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda8015bdc0 0x7fda8014cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47368/0 (socket says 172.21.15.26:47368) 2024-04-26T21:09:59.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.540+0000 7fda85000640 1 -- 172.21.15.26:0/1453315838 learned_addr learned my addr 172.21.15.26:0/1453315838 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:09:59.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.540+0000 7fda77e00640 1 --2- 172.21.15.26:0/1453315838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda80073530 0x7fda8014c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:59.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.540+0000 7fda7fe00640 1 --2- 172.21.15.26:0/1453315838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda8006e210 0x7fda8013fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:59.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.540+0000 7fda85000640 1 -- 172.21.15.26:0/1453315838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda80073530 msgr2=0x7fda8014c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:59.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.541+0000 7fda85000640 1 --2- 172.21.15.26:0/1453315838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda80073530 0x7fda8014c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.541+0000 7fda85000640 1 -- 172.21.15.26:0/1453315838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda8006e210 msgr2=0x7fda8013fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:59.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.541+0000 7fda85000640 1 --2- 172.21.15.26:0/1453315838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda8006e210 0x7fda8013fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.541+0000 7fda85000640 1 -- 172.21.15.26:0/1453315838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8014d640 con 0x7fda8015bdc0 2024-04-26T21:09:59.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.541+0000 7fda85000640 1 --2- 172.21.15.26:0/1453315838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda8015bdc0 0x7fda8014cd50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fda7800ce80 tx=0x7fda7800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:59.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.541+0000 7fda7cc00640 1 -- 172.21.15.26:0/1453315838 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7801a070 con 0x7fda8015bdc0 2024-04-26T21:09:59.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.541+0000 7fda7cc00640 1 -- 172.21.15.26:0/1453315838 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda7800d880 con 0x7fda8015bdc0 2024-04-26T21:09:59.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.541+0000 7fda7cc00640 1 -- 172.21.15.26:0/1453315838 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7800dc70 con 0x7fda8015bdc0 2024-04-26T21:09:59.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.541+0000 7fda85a00640 1 -- 172.21.15.26:0/1453315838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda8014d920 con 0x7fda8015bdc0 2024-04-26T21:09:59.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.541+0000 7fda85a00640 1 -- 172.21.15.26:0/1453315838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda8014dee0 con 0x7fda8015bdc0 2024-04-26T21:09:59.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.543+0000 7fda85a00640 1 -- 172.21.15.26:0/1453315838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda8006a700 con 0x7fda8015bdc0 2024-04-26T21:09:59.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.544+0000 7fda7cc00640 1 -- 172.21.15.26:0/1453315838 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda7801d070 con 0x7fda8015bdc0 2024-04-26T21:09:59.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.544+0000 7fda7cc00640 1 --2- 172.21.15.26:0/1453315838 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fda5c06c8f0 0x7fda5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:09:59.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.545+0000 7fda7cc00640 1 -- 172.21.15.26:0/1453315838 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fda7808c330 con 0x7fda8015bdc0 2024-04-26T21:09:59.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.545+0000 7fda7fe00640 1 --2- 172.21.15.26:0/1453315838 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fda5c06c8f0 0x7fda5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:09:59.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.545+0000 7fda7fe00640 1 --2- 172.21.15.26:0/1453315838 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fda5c06c8f0 0x7fda5c06edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fda70034000 tx=0x7fda7002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:09:59.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.549+0000 7fda7cc00640 1 -- 172.21.15.26:0/1453315838 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda7805a070 con 0x7fda8015bdc0 2024-04-26T21:09:59.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.753+0000 7fda85a00640 1 -- 172.21.15.26:0/1453315838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mds_symlink_recovery}] v 0) -- 0x7fda800d0610 con 0x7fda8015bdc0 2024-04-26T21:09:59.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.761+0000 7fda7cc00640 1 -- 172.21.15.26:0/1453315838 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_symlink_recovery}]=0 v12)=0 v12) ==== 121+0+0 (secure 0 0 0) 0x7fda7805e370 con 0x7fda8015bdc0 2024-04-26T21:09:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.764+0000 7fda85a00640 1 -- 172.21.15.26:0/1453315838 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fda5c06c8f0 msgr2=0x7fda5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.764+0000 7fda85a00640 1 --2- 172.21.15.26:0/1453315838 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fda5c06c8f0 0x7fda5c06edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fda70034000 tx=0x7fda7002e000 comp rx=0 tx=0).stop 2024-04-26T21:09:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.764+0000 7fda85a00640 1 -- 172.21.15.26:0/1453315838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda8015bdc0 msgr2=0x7fda8014cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:09:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.764+0000 7fda85a00640 1 --2- 172.21.15.26:0/1453315838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda8015bdc0 0x7fda8014cd50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fda7800ce80 tx=0x7fda7800a710 comp rx=0 tx=0).stop 2024-04-26T21:09:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.764+0000 7fda85a00640 1 -- 172.21.15.26:0/1453315838 shutdown_connections 2024-04-26T21:09:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.764+0000 7fda85a00640 1 --2- 172.21.15.26:0/1453315838 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fda5c06c8f0 0x7fda5c06edb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.764+0000 7fda85a00640 1 --2- 172.21.15.26:0/1453315838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda8015bdc0 0x7fda8014cd50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.764+0000 7fda85a00640 1 --2- 172.21.15.26:0/1453315838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda80073530 0x7fda8014c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.764+0000 7fda85a00640 1 --2- 172.21.15.26:0/1453315838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda8006e210 0x7fda8013fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:09:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.764+0000 7fda85a00640 1 -- 172.21.15.26:0/1453315838 >> 172.21.15.26:0/1453315838 conn(0x7fda800b2a10 msgr2=0x7fda8006ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:09:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.765+0000 7fda85a00640 1 -- 172.21.15.26:0/1453315838 shutdown_connections 2024-04-26T21:09:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:09:59.765+0000 7fda85a00640 1 -- 172.21.15.26:0/1453315838 wait complete. 2024-04-26T21:09:59.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:59.786 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir sym_dir) 2024-04-26T21:09:59.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:59.842 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch sym_dir/file1) 2024-04-26T21:09:59.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:09:59.902 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln -s ./file1 sym_dir/symlink_file1) 2024-04-26T21:09:59.961 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-26T21:09:59.961 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-26T21:09:59.962 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-26T21:09:59.962 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-26T21:09:59.962 DEBUG:teuthology.orchestra.run.smithi026:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/sym_dir/symlink_file1").st_ino) 2024-04-26T21:09:59.962 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-26T21:10:00.096 INFO:teuthology.orchestra.run.smithi026.stdout:1099511628281 2024-04-26T21:10:00.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:10:00.111 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:10:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.371+0000 7f599aa00640 1 Processor -- start 2024-04-26T21:10:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.372+0000 7f599aa00640 1 -- start start 2024-04-26T21:10:00.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.373+0000 7f599aa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59940d1840 0x7f5994071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:00.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.373+0000 7f599aa00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59940cf690 con 0x7f59940d06f0 2024-04-26T21:10:00.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.373+0000 7f599aa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59940cf930 con 0x7f5994145c20 2024-04-26T21:10:00.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.373+0000 7f599aa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f59940ced10 con 0x7f59940d1840 2024-04-26T21:10:00.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.374+0000 7f5998c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f59940d06f0 0x7f59940d1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55450/0 (socket says 172.21.15.26:55450) 2024-04-26T21:10:00.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.374+0000 7f5998c00640 1 -- 172.21.15.26:0/62429266 learned_addr learned my addr 172.21.15.26:0/62429266 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.374+0000 7f599a000640 1 --2- 172.21.15.26:0/62429266 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59940d1840 0x7f5994071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.374+0000 7f5993e00640 1 -- 172.21.15.26:0/62429266 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2737739054 0 0) 0x7f59940cf690 con 0x7f59940d06f0 2024-04-26T21:10:00.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.374+0000 7f5993e00640 1 -- 172.21.15.26:0/62429266 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f597c003520 con 0x7f59940d06f0 2024-04-26T21:10:00.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.375+0000 7f5993e00640 1 -- 172.21.15.26:0/62429266 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3589458207 0 0) 0x7f597c003520 con 0x7f59940d06f0 2024-04-26T21:10:00.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.375+0000 7f5993e00640 1 -- 172.21.15.26:0/62429266 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59940d1840 msgr2=0x7f5994071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:00.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.375+0000 7f5993e00640 1 --2- 172.21.15.26:0/62429266 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59940d1840 0x7f5994071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:00.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.375+0000 7f5993e00640 1 -- 172.21.15.26:0/62429266 >> v1:172.21.15.86:6789/0 conn(0x7f5994145c20 legacy=0x7f5994146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:00.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.375+0000 7f5993e00640 1 -- 172.21.15.26:0/62429266 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59940cfc10 con 0x7f59940d06f0 2024-04-26T21:10:00.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.375+0000 7f5993e00640 1 -- 172.21.15.26:0/62429266 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f5980001670 con 0x7f59940d06f0 2024-04-26T21:10:00.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.375+0000 7f5993e00640 1 -- 172.21.15.26:0/62429266 >> v1:172.21.15.26:6789/0 conn(0x7f59940d06f0 legacy=0x7f59940d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:00.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.375+0000 7f5993e00640 1 --2- 172.21.15.26:0/62429266 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59940d1840 0x7f597c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:00.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.375+0000 7f5993e00640 1 --2- 172.21.15.26:0/62429266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f597c003f60 0x7f597c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:00.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.375+0000 7f5993e00640 1 --2- 172.21.15.26:0/62429266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f597c004d40 0x7f597c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:00.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.375+0000 7f5993e00640 1 -- 172.21.15.26:0/62429266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f597c00ba70 con 0x7f597c004d40 2024-04-26T21:10:00.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.375+0000 7f5993e00640 1 -- 172.21.15.26:0/62429266 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f597c00bc20 con 0x7f597c003f60 2024-04-26T21:10:00.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.375+0000 7f5993e00640 1 -- 172.21.15.26:0/62429266 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f597c00be20 con 0x7f59940d1840 2024-04-26T21:10:00.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.375+0000 7f599a000640 1 --2- 172.21.15.26:0/62429266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f597c004d40 0x7f597c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:00.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.375+0000 7f5999600640 1 --2- 172.21.15.26:0/62429266 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59940d1840 0x7f597c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:00.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.375+0000 7f599a000640 1 -- 172.21.15.26:0/62429266 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59940d1840 msgr2=0x7f597c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:00.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.376+0000 7f599a000640 1 --2- 172.21.15.26:0/62429266 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59940d1840 0x7f597c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:00.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.376+0000 7f599a000640 1 -- 172.21.15.26:0/62429266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f597c003f60 msgr2=0x7f597c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:00.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.376+0000 7f599a000640 1 --2- 172.21.15.26:0/62429266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f597c003f60 0x7f597c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:00.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.376+0000 7f599a000640 1 -- 172.21.15.26:0/62429266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c003520 con 0x7f597c004d40 2024-04-26T21:10:00.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.376+0000 7f5999600640 1 --2- 172.21.15.26:0/62429266 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59940d1840 0x7f597c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:00.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.377+0000 7f599a000640 1 --2- 172.21.15.26:0/62429266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f597c004d40 0x7f597c0051c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f598c009fc0 tx=0x7f597c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1a0f7e1f7850b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:00.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.377+0000 7f5993e00640 1 -- 172.21.15.26:0/62429266 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c009c10 con 0x7f597c004d40 2024-04-26T21:10:00.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.377+0000 7f5993e00640 1 -- 172.21.15.26:0/62429266 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f598c036070 con 0x7f597c004d40 2024-04-26T21:10:00.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.377+0000 7f5993e00640 1 -- 172.21.15.26:0/62429266 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c031860 con 0x7f597c004d40 2024-04-26T21:10:00.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.377+0000 7f599aa00640 1 -- 172.21.15.26:0/62429266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f597c004d40 msgr2=0x7f597c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:00.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.377+0000 7f599aa00640 1 --2- 172.21.15.26:0/62429266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f597c004d40 0x7f597c0051c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f598c009fc0 tx=0x7f597c005830 comp rx=0 tx=0).stop 2024-04-26T21:10:00.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.377+0000 7f599a000640 1 -- 172.21.15.26:0/62429266 reap_dead start 2024-04-26T21:10:00.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.378+0000 7f599aa00640 1 -- 172.21.15.26:0/62429266 shutdown_connections 2024-04-26T21:10:00.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.378+0000 7f599aa00640 1 -- 172.21.15.26:0/62429266 >> 172.21.15.26:0/62429266 conn(0x7f599406c690 msgr2=0x7f599406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:00.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.378+0000 7f599aa00640 1 -- 172.21.15.26:0/62429266 shutdown_connections 2024-04-26T21:10:00.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.378+0000 7f599aa00640 1 -- 172.21.15.26:0/62429266 wait complete. 2024-04-26T21:10:00.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.380+0000 7f599aa00640 1 Processor -- start 2024-04-26T21:10:00.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.380+0000 7f599aa00640 1 -- start start 2024-04-26T21:10:00.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.381+0000 7f599aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f597c004d40 0x7f5994141540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:00.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.381+0000 7f599aa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f599415a540 0x7f599415a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:00.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.381+0000 7f599aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59940af8a0 0x7f59940afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:00.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.381+0000 7f599aa00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f59940d01d0 con 0x7f597c004d40 2024-04-26T21:10:00.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.381+0000 7f599aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5994146b60 con 0x7f59940af8a0 2024-04-26T21:10:00.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.381+0000 7f599aa00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5994146000 con 0x7f599415a540 2024-04-26T21:10:00.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.381+0000 7f5999600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f597c004d40 0x7f5994141540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:00.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.381+0000 7f5999600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f597c004d40 0x7f5994141540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47396/0 (socket says 172.21.15.26:47396) 2024-04-26T21:10:00.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.381+0000 7f5999600640 1 -- 172.21.15.26:0/3924440998 learned_addr learned my addr 172.21.15.26:0/3924440998 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:00.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.381+0000 7f5998c00640 1 --2- 172.21.15.26:0/3924440998 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f599415a540 0x7f599415a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:00.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.381+0000 7f599a000640 1 --2- 172.21.15.26:0/3924440998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59940af8a0 0x7f59940afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:00.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.381+0000 7f5999600640 1 -- 172.21.15.26:0/3924440998 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f599415a540 msgr2=0x7f599415a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:00.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.381+0000 7f5999600640 1 --2- 172.21.15.26:0/3924440998 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f599415a540 0x7f599415a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:00.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.381+0000 7f5999600640 1 -- 172.21.15.26:0/3924440998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59940af8a0 msgr2=0x7f59940afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:00.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.381+0000 7f5999600640 1 --2- 172.21.15.26:0/3924440998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59940af8a0 0x7f59940afd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:00.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.381+0000 7f5999600640 1 -- 172.21.15.26:0/3924440998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599415b0e0 con 0x7f597c004d40 2024-04-26T21:10:00.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.381+0000 7f5998c00640 1 --2- 172.21.15.26:0/3924440998 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f599415a540 0x7f599415a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:00.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.382+0000 7f5999600640 1 --2- 172.21.15.26:0/3924440998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f597c004d40 0x7f5994141540 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f598800d7a0 tx=0x7f598800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:00.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.382+0000 7f5991600640 1 -- 172.21.15.26:0/3924440998 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5988005070 con 0x7f597c004d40 2024-04-26T21:10:00.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.382+0000 7f5991600640 1 -- 172.21.15.26:0/3924440998 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5988005210 con 0x7f597c004d40 2024-04-26T21:10:00.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.382+0000 7f5991600640 1 -- 172.21.15.26:0/3924440998 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5988007030 con 0x7f597c004d40 2024-04-26T21:10:00.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.382+0000 7f599aa00640 1 -- 172.21.15.26:0/3924440998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f599415b3c0 con 0x7f597c004d40 2024-04-26T21:10:00.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.382+0000 7f599aa00640 1 -- 172.21.15.26:0/3924440998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f599415b980 con 0x7f597c004d40 2024-04-26T21:10:00.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.383+0000 7f599aa00640 1 -- 172.21.15.26:0/3924440998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59940b0260 con 0x7f597c004d40 2024-04-26T21:10:00.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.384+0000 7f5991600640 1 -- 172.21.15.26:0/3924440998 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f598801a020 con 0x7f597c004d40 2024-04-26T21:10:00.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.385+0000 7f5991600640 1 --2- 172.21.15.26:0/3924440998 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f597406c920 0x7f597406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:00.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.385+0000 7f5991600640 1 -- 172.21.15.26:0/3924440998 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f598808b420 con 0x7f597c004d40 2024-04-26T21:10:00.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.386+0000 7f5998c00640 1 --2- 172.21.15.26:0/3924440998 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f597406c920 0x7f597406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:00.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.386+0000 7f5998c00640 1 --2- 172.21.15.26:0/3924440998 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f597406c920 0x7f597406ede0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5980001eb0 tx=0x7f5980009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:00.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.390+0000 7f5991600640 1 -- 172.21.15.26:0/3924440998 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5988059840 con 0x7f597c004d40 2024-04-26T21:10:00.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.632+0000 7f599aa00640 1 -- 172.21.15.26:0/3924440998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f599406a700 con 0x7f597c004d40 2024-04-26T21:10:00.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.634+0000 7f5991600640 1 -- 172.21.15.26:0/3924440998 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7f5988059140 con 0x7f597c004d40 2024-04-26T21:10:00.635 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:00.636 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2349160700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2349160700},{"type":"v1","addr":"172.21.15.26:6837","nonce":2349160700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4598,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2824191762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2824191762},{"type":"v1","addr":"172.21.15.86:6834","nonce":2824191762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/263746310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":263746310},{"type":"v1","addr":"172.21.15.86:6835","nonce":263746310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:09:48.622025+0000","modified":"2024-04-26T21:09:51.721103+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5058":{"gid":5058,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6836/2528180854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2528180854},{"type":"v1","addr":"172.21.15.26:6836","nonce":2528180854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5058,"qdb_cluster":[5058]},"id":2}]} 2024-04-26T21:10:00.636 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 17 2024-04-26T21:10:00.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.636+0000 7f599aa00640 1 -- 172.21.15.26:0/3924440998 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f597406c920 msgr2=0x7f597406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:00.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.636+0000 7f599aa00640 1 --2- 172.21.15.26:0/3924440998 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f597406c920 0x7f597406ede0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5980001eb0 tx=0x7f5980009040 comp rx=0 tx=0).stop 2024-04-26T21:10:00.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.636+0000 7f599aa00640 1 -- 172.21.15.26:0/3924440998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f597c004d40 msgr2=0x7f5994141540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:00.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.636+0000 7f599aa00640 1 --2- 172.21.15.26:0/3924440998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f597c004d40 0x7f5994141540 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f598800d7a0 tx=0x7f598800dc60 comp rx=0 tx=0).stop 2024-04-26T21:10:00.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.636+0000 7f599aa00640 1 -- 172.21.15.26:0/3924440998 shutdown_connections 2024-04-26T21:10:00.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.636+0000 7f599aa00640 1 --2- 172.21.15.26:0/3924440998 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f597406c920 0x7f597406ede0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:00.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.636+0000 7f599aa00640 1 --2- 172.21.15.26:0/3924440998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59940af8a0 0x7f59940afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:00.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.636+0000 7f599aa00640 1 --2- 172.21.15.26:0/3924440998 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f599415a540 0x7f599415a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:00.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.636+0000 7f599aa00640 1 --2- 172.21.15.26:0/3924440998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f597c004d40 0x7f5994141540 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:00.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.636+0000 7f599aa00640 1 -- 172.21.15.26:0/3924440998 >> 172.21.15.26:0/3924440998 conn(0x7f599406c690 msgr2=0x7f59940b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:00.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.637+0000 7f599aa00640 1 -- 172.21.15.26:0/3924440998 shutdown_connections 2024-04-26T21:10:00.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:00.637+0000 7f599aa00640 1 -- 172.21.15.26:0/3924440998 wait complete. 2024-04-26T21:10:00.658 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-26T21:10:00.908 INFO:teuthology.orchestra.run.smithi026.stdout:{"message":"","return_code":0} 2024-04-26T21:10:00.909 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-26T21:10:00.909 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f9.00000000 symlink 2024-04-26T21:10:01.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.001+0000 7fba6f340f40 1 Processor -- start 2024-04-26T21:10:01.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.001+0000 7fba6f340f40 1 -- start start 2024-04-26T21:10:01.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.001+0000 7fba6f340f40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558e575b8f50 0x558e575b6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:01.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.001+0000 7fba6f340f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e57646050 0x558e5756c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:01.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.001+0000 7fba6f340f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e57648eb0 0x558e5759cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:01.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.001+0000 7fba6f340f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558e57646600 con 0x558e57646050 2024-04-26T21:10:01.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.001+0000 7fba6f340f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558e575bbe70 con 0x558e57648eb0 2024-04-26T21:10:01.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.001+0000 7fba6f340f40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x558e57646b80 con 0x558e575b8f50 2024-04-26T21:10:01.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.002+0000 7fba6bc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e57646050 0x558e5756c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:01.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.002+0000 7fba6bc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e57646050 0x558e5756c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47408/0 (socket says 172.21.15.26:47408) 2024-04-26T21:10:01.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.002+0000 7fba6bc00640 1 -- 172.21.15.26:0/2595155551 learned_addr learned my addr 172.21.15.26:0/2595155551 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:01.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.002+0000 7fba6c600640 1 --2- 172.21.15.26:0/2595155551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558e575b8f50 0x558e575b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:01.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.002+0000 7fba6bc00640 1 -- 172.21.15.26:0/2595155551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558e575b8f50 msgr2=0x558e575b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:01.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.002+0000 7fba6bc00640 1 --2- 172.21.15.26:0/2595155551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558e575b8f50 0x558e575b6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:01.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.002+0000 7fba6bc00640 1 -- 172.21.15.26:0/2595155551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e57648eb0 msgr2=0x558e5759cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:01.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.002+0000 7fba6d000640 1 --2- 172.21.15.26:0/2595155551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e57648eb0 0x558e5759cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:01.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.002+0000 7fba6bc00640 1 --2- 172.21.15.26:0/2595155551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e57648eb0 0x558e5759cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:01.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.002+0000 7fba6bc00640 1 -- 172.21.15.26:0/2595155551 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e57498b30 con 0x558e57646050 2024-04-26T21:10:01.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.003+0000 7fba6bc00640 1 --2- 172.21.15.26:0/2595155551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e57646050 0x558e5756c2d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fba58000c00 tx=0x7fba58051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cbe261f3c7a99bf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:01.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.003+0000 7fba6a800640 1 -- 172.21.15.26:0/2595155551 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba58062070 con 0x558e57646050 2024-04-26T21:10:01.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.003+0000 7fba6a800640 1 -- 172.21.15.26:0/2595155551 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba58054970 con 0x558e57646050 2024-04-26T21:10:01.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.003+0000 7fba6a800640 1 -- 172.21.15.26:0/2595155551 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba58054d20 con 0x558e57646050 2024-04-26T21:10:01.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.003+0000 7fba6f340f40 1 -- 172.21.15.26:0/2595155551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e57646050 msgr2=0x558e5756c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:01.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.003+0000 7fba6f340f40 1 --2- 172.21.15.26:0/2595155551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e57646050 0x558e5756c2d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fba58000c00 tx=0x7fba58051000 comp rx=0 tx=0).stop 2024-04-26T21:10:01.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.004+0000 7fba6f340f40 1 -- 172.21.15.26:0/2595155551 shutdown_connections 2024-04-26T21:10:01.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.004+0000 7fba6f340f40 1 --2- 172.21.15.26:0/2595155551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e57648eb0 0x558e5759cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:01.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.004+0000 7fba6f340f40 1 --2- 172.21.15.26:0/2595155551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e57646050 0x558e5756c2d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:01.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.004+0000 7fba6f340f40 1 --2- 172.21.15.26:0/2595155551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558e575b8f50 0x558e575b6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:01.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.004+0000 7fba6f340f40 1 -- 172.21.15.26:0/2595155551 >> 172.21.15.26:0/2595155551 conn(0x558e5759e100 msgr2=0x558e576484e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:01.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.004+0000 7fba6f340f40 1 -- 172.21.15.26:0/2595155551 shutdown_connections 2024-04-26T21:10:01.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.004+0000 7fba6f340f40 1 -- 172.21.15.26:0/2595155551 wait complete. 2024-04-26T21:10:01.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.004+0000 7fba6f340f40 1 Processor -- start 2024-04-26T21:10:01.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.005+0000 7fba6f340f40 1 -- start start 2024-04-26T21:10:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.006+0000 7fba6f340f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e575b8f50 0x558e576383b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.006+0000 7fba6f340f40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558e57646050 0x558e5763d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.006+0000 7fba6f340f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e57648eb0 0x558e5759cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.006+0000 7fba6f340f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x558e57640190 con 0x558e575b8f50 2024-04-26T21:10:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.006+0000 7fba6f340f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558e5763fb20 con 0x558e57648eb0 2024-04-26T21:10:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.006+0000 7fba6f340f40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x558e57647940 con 0x558e57646050 2024-04-26T21:10:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.006+0000 7fba6c600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e575b8f50 0x558e576383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.006+0000 7fba6c600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e575b8f50 0x558e576383b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47424/0 (socket says 172.21.15.26:47424) 2024-04-26T21:10:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.006+0000 7fba6c600640 1 -- 172.21.15.26:0/642071696 learned_addr learned my addr 172.21.15.26:0/642071696 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.006+0000 7fba6c600640 1 -- 172.21.15.26:0/642071696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558e57646050 msgr2=0x558e5763d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.006+0000 7fba6c600640 1 --2- 172.21.15.26:0/642071696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x558e57646050 0x558e5763d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.006+0000 7fba6c600640 1 -- 172.21.15.26:0/642071696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e57648eb0 msgr2=0x558e5759cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:01.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.006+0000 7fba6c600640 1 --2- 172.21.15.26:0/642071696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e57648eb0 0x558e5759cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.006+0000 7fba6c600640 1 -- 172.21.15.26:0/642071696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e5763e6a0 con 0x558e575b8f50 2024-04-26T21:10:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.007+0000 7fba6c600640 1 --2- 172.21.15.26:0/642071696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e575b8f50 0x558e576383b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00be10 tx=0x7fba5c00bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.007+0000 7fba69400640 1 -- 172.21.15.26:0/642071696 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c007b50 con 0x558e575b8f50 2024-04-26T21:10:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.007+0000 7fba69400640 1 -- 172.21.15.26:0/642071696 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba5c007cf0 con 0x558e575b8f50 2024-04-26T21:10:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.007+0000 7fba69400640 1 -- 172.21.15.26:0/642071696 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c015690 con 0x558e575b8f50 2024-04-26T21:10:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.007+0000 7fba6f340f40 1 -- 172.21.15.26:0/642071696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558e57639440 con 0x558e575b8f50 2024-04-26T21:10:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.007+0000 7fba6f340f40 1 -- 172.21.15.26:0/642071696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e5763de00 con 0x558e575b8f50 2024-04-26T21:10:01.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.009+0000 7fba69400640 1 -- 172.21.15.26:0/642071696 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba5c015830 con 0x558e575b8f50 2024-04-26T21:10:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.010+0000 7fba69400640 1 --2- 172.21.15.26:0/642071696 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fba4c06b2d0 0x7fba4c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:01.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.010+0000 7fba69400640 1 -- 172.21.15.26:0/642071696 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fba5c089e40 con 0x558e575b8f50 2024-04-26T21:10:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.010+0000 7fba6bc00640 1 --2- 172.21.15.26:0/642071696 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fba4c06b2d0 0x7fba4c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.010+0000 7fba6f340f40 1 --2- 172.21.15.26:0/642071696 >> [v2:172.21.15.26:6800/398686833,v1:172.21.15.26:6801/398686833] conn(0x558e5763d5e0 0x558e57639e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.010+0000 7fba6f340f40 1 -- 172.21.15.26:0/642071696 --> [v2:172.21.15.26:6800/398686833,v1:172.21.15.26:6801/398686833] -- osd_op(unknown.0.0:1 5.36 5:6ee5e2c4:::100000001f9.00000000:head [getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) -- 0x558e5763a370 con 0x558e5763d5e0 2024-04-26T21:10:01.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.011+0000 7fba6d000640 1 --2- 172.21.15.26:0/642071696 >> [v2:172.21.15.26:6800/398686833,v1:172.21.15.26:6801/398686833] conn(0x558e5763d5e0 0x558e57639e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.011+0000 7fba6bc00640 1 --2- 172.21.15.26:0/642071696 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fba4c06b2d0 0x7fba4c06d790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fba58066000 tx=0x7fba58005ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.011+0000 7fba6d000640 1 --2- 172.21.15.26:0/642071696 >> [v2:172.21.15.26:6800/398686833,v1:172.21.15.26:6801/398686833] conn(0x558e5763d5e0 0x558e57639e30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.012+0000 7fba6d000640 1 -- 172.21.15.26:0/642071696 <== osd.1 v2:172.21.15.26:6800/398686833 1 ==== osd_op_reply(1 100000001f9.00000000 [getxattr (11) out=11b] v0'0 uv1 ondisk = 0) ==== 164+0+11 (crc 0 0 0) 0x7fba6400c560 con 0x558e5763d5e0 2024-04-26T21:10:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.012+0000 7fba6f340f40 1 -- 172.21.15.26:0/642071696 >> [v2:172.21.15.26:6800/398686833,v1:172.21.15.26:6801/398686833] conn(0x558e5763d5e0 msgr2=0x558e57639e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.012+0000 7fba6f340f40 1 --2- 172.21.15.26:0/642071696 >> [v2:172.21.15.26:6800/398686833,v1:172.21.15.26:6801/398686833] conn(0x558e5763d5e0 0x558e57639e30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.012+0000 7fba6f340f40 1 -- 172.21.15.26:0/642071696 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fba4c06b2d0 msgr2=0x7fba4c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.012+0000 7fba6f340f40 1 --2- 172.21.15.26:0/642071696 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fba4c06b2d0 0x7fba4c06d790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fba58066000 tx=0x7fba58005ac0 comp rx=0 tx=0).stop 2024-04-26T21:10:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.012+0000 7fba6f340f40 1 -- 172.21.15.26:0/642071696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e575b8f50 msgr2=0x558e576383b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.012+0000 7fba6f340f40 1 --2- 172.21.15.26:0/642071696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558e575b8f50 0x558e576383b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00be10 tx=0x7fba5c00bf00 comp rx=0 tx=0).stop 2024-04-26T21:10:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.012+0000 7fba6d000640 1 -- 172.21.15.26:0/642071696 reap_dead start 2024-04-26T21:10:01.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.012+0000 7fba6f340f40 1 -- 172.21.15.26:0/642071696 shutdown_connections 2024-04-26T21:10:01.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.012+0000 7fba6f340f40 1 -- 172.21.15.26:0/642071696 >> 172.21.15.26:0/642071696 conn(0x558e5759e100 msgr2=0x558e5755c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:01.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.012+0000 7fba6f340f40 1 -- 172.21.15.26:0/642071696 shutdown_connections 2024-04-26T21:10:01.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.012+0000 7fba6f340f40 1 -- 172.21.15.26:0/642071696 wait complete. 2024-04-26T21:10:01.018 INFO:teuthology.orchestra.run.smithi026.stdout:./file1 2024-04-26T21:10:01.018 DEBUG:teuthology.orchestra.run.smithi026:> ceph-dencoder type string_wrapper import - decode dump_json 2024-04-26T21:10:01.162 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-26T21:10:01.162 INFO:teuthology.orchestra.run.smithi026.stdout: "s": "./file1" 2024-04-26T21:10:01.162 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-26T21:10:01.162 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:01.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:10:01.172 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-04-26T21:10:01.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:10:01.228 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-04-26T21:10:01.289 DEBUG:teuthology.orchestra.run.smithi026:> stdin-killer -- python3 -c ' 2024-04-26T21:10:01.290 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-04-26T21:10:01.290 DEBUG:teuthology.orchestra.run.smithi026:> import stat 2024-04-26T21:10:01.290 DEBUG:teuthology.orchestra.run.smithi026:> 2024-04-26T21:10:01.290 DEBUG:teuthology.orchestra.run.smithi026:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-04-26T21:10:01.290 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-04-26T21:10:01.437 INFO:teuthology.orchestra.run.smithi026.stdout:1099511628283 2024-04-26T21:10:01.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:10:01.451 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:10:01.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.708+0000 7f1dc2c00640 1 Processor -- start 2024-04-26T21:10:01.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.708+0000 7f1dc2c00640 1 -- start start 2024-04-26T21:10:01.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.709+0000 7f1dc2c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dbc15ffa0 0x7f1dbc0d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:01.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.709+0000 7f1dc2c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dbc160440 con 0x7f1dbc159770 2024-04-26T21:10:01.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.709+0000 7f1dc2c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dbc160720 con 0x7f1dbc159b50 2024-04-26T21:10:01.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.709+0000 7f1dc2c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1dbc072560 con 0x7f1dbc15ffa0 2024-04-26T21:10:01.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.710+0000 7f1dc1800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1dbc159770 0x7f1dbc0b4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55454/0 (socket says 172.21.15.26:55454) 2024-04-26T21:10:01.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.710+0000 7f1dc1800640 1 -- 172.21.15.26:0/3245312175 learned_addr learned my addr 172.21.15.26:0/3245312175 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.710+0000 7f1dc2200640 1 --2- 172.21.15.26:0/3245312175 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dbc15ffa0 0x7f1dbc0d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.710+0000 7f1dbbe00640 1 -- 172.21.15.26:0/3245312175 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 674262749 0 0) 0x7f1dbc160440 con 0x7f1dbc159770 2024-04-26T21:10:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.711+0000 7f1dbbe00640 1 -- 172.21.15.26:0/3245312175 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1da4003520 con 0x7f1dbc159770 2024-04-26T21:10:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.711+0000 7f1dbbe00640 1 -- 172.21.15.26:0/3245312175 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1581499043 0 0) 0x7f1da4003520 con 0x7f1dbc159770 2024-04-26T21:10:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.711+0000 7f1dbbe00640 1 -- 172.21.15.26:0/3245312175 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dbc15ffa0 msgr2=0x7f1dbc0d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.711+0000 7f1dbbe00640 1 --2- 172.21.15.26:0/3245312175 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dbc15ffa0 0x7f1dbc0d1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.711+0000 7f1dbbe00640 1 -- 172.21.15.26:0/3245312175 >> v1:172.21.15.86:6789/0 conn(0x7f1dbc159b50 legacy=0x7f1dbc15f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.711+0000 7f1dbbe00640 1 -- 172.21.15.26:0/3245312175 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dbc072250 con 0x7f1dbc159770 2024-04-26T21:10:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.711+0000 7f1dbbe00640 1 -- 172.21.15.26:0/3245312175 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f1db00019f0 con 0x7f1dbc159770 2024-04-26T21:10:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.711+0000 7f1dbbe00640 1 -- 172.21.15.26:0/3245312175 >> v1:172.21.15.26:6789/0 conn(0x7f1dbc159770 legacy=0x7f1dbc0b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:01.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.711+0000 7f1dbbe00640 1 --2- 172.21.15.26:0/3245312175 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dbc15ffa0 0x7f1da40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.711+0000 7f1dbbe00640 1 --2- 172.21.15.26:0/3245312175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1da4003fb0 0x7f1da4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.711+0000 7f1dbbe00640 1 --2- 172.21.15.26:0/3245312175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1da4004d40 0x7f1da40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.711+0000 7f1dbbe00640 1 -- 172.21.15.26:0/3245312175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc072560 con 0x7f1da4004d40 2024-04-26T21:10:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.711+0000 7f1dbbe00640 1 -- 172.21.15.26:0/3245312175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1da400bbb0 con 0x7f1da4003fb0 2024-04-26T21:10:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.711+0000 7f1dbbe00640 1 -- 172.21.15.26:0/3245312175 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1da400bdb0 con 0x7f1dbc15ffa0 2024-04-26T21:10:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.712+0000 7f1dc2200640 1 --2- 172.21.15.26:0/3245312175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1da4004d40 0x7f1da40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.712+0000 7f1dc1800640 1 --2- 172.21.15.26:0/3245312175 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dbc15ffa0 0x7f1da40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.712+0000 7f1dc0e00640 1 --2- 172.21.15.26:0/3245312175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1da4003fb0 0x7f1da4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.712+0000 7f1dc2200640 1 -- 172.21.15.26:0/3245312175 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dbc15ffa0 msgr2=0x7f1da40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.712+0000 7f1dc2200640 1 --2- 172.21.15.26:0/3245312175 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dbc15ffa0 0x7f1da40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.712+0000 7f1dc2200640 1 -- 172.21.15.26:0/3245312175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1da4003fb0 msgr2=0x7f1da4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.712+0000 7f1dc2200640 1 --2- 172.21.15.26:0/3245312175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1da4003fb0 0x7f1da4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.712+0000 7f1dc2200640 1 -- 172.21.15.26:0/3245312175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da4003520 con 0x7f1da4004d40 2024-04-26T21:10:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.712+0000 7f1dc0e00640 1 --2- 172.21.15.26:0/3245312175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1da4003fb0 0x7f1da4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:01.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.713+0000 7f1dc2200640 1 --2- 172.21.15.26:0/3245312175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1da4004d40 0x7f1da40051c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009ac0 tx=0x7f1da4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5ad246abdc82db server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:01.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.714+0000 7f1dbbe00640 1 -- 172.21.15.26:0/3245312175 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4036070 con 0x7f1da4004d40 2024-04-26T21:10:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.714+0000 7f1dbbe00640 1 -- 172.21.15.26:0/3245312175 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db4031400 con 0x7f1da4004d40 2024-04-26T21:10:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.714+0000 7f1dbbe00640 1 -- 172.21.15.26:0/3245312175 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4009690 con 0x7f1da4004d40 2024-04-26T21:10:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.714+0000 7f1dc2c00640 1 -- 172.21.15.26:0/3245312175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1da4004d40 msgr2=0x7f1da40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.714+0000 7f1dc2c00640 1 --2- 172.21.15.26:0/3245312175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1da4004d40 0x7f1da40051c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009ac0 tx=0x7f1da4005830 comp rx=0 tx=0).stop 2024-04-26T21:10:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.714+0000 7f1dc2200640 1 -- 172.21.15.26:0/3245312175 reap_dead start 2024-04-26T21:10:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.714+0000 7f1dc2c00640 1 -- 172.21.15.26:0/3245312175 shutdown_connections 2024-04-26T21:10:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.714+0000 7f1dc2c00640 1 -- 172.21.15.26:0/3245312175 >> 172.21.15.26:0/3245312175 conn(0x7f1dbc06c690 msgr2=0x7f1dbc06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.714+0000 7f1dc2c00640 1 -- 172.21.15.26:0/3245312175 shutdown_connections 2024-04-26T21:10:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.714+0000 7f1dc2c00640 1 -- 172.21.15.26:0/3245312175 wait complete. 2024-04-26T21:10:01.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.716+0000 7f1dc2c00640 1 Processor -- start 2024-04-26T21:10:01.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.716+0000 7f1dc2c00640 1 -- start start 2024-04-26T21:10:01.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.717+0000 7f1dc2c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1da4004d40 0x7f1dbc1499a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:01.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.717+0000 7f1dc2c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dbc146f90 0x7f1dbc15e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:01.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.717+0000 7f1dc2c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dbc156280 0x7f1dbc15c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:01.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.717+0000 7f1dc2c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc0d12b0 con 0x7f1dbc156280 2024-04-26T21:10:01.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.717+0000 7f1dc2c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc0d1130 con 0x7f1da4004d40 2024-04-26T21:10:01.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.717+0000 7f1dc2c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f1dbc0d1430 con 0x7f1dbc146f90 2024-04-26T21:10:01.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.718+0000 7f1dc2200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dbc156280 0x7f1dbc15c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:01.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.718+0000 7f1dc2200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dbc156280 0x7f1dbc15c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47434/0 (socket says 172.21.15.26:47434) 2024-04-26T21:10:01.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.718+0000 7f1dc2200640 1 -- 172.21.15.26:0/2893783814 learned_addr learned my addr 172.21.15.26:0/2893783814 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:01.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.718+0000 7f1dc0e00640 1 --2- 172.21.15.26:0/2893783814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dbc146f90 0x7f1dbc15e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:01.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.718+0000 7f1dc1800640 1 --2- 172.21.15.26:0/2893783814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1da4004d40 0x7f1dbc1499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:01.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.718+0000 7f1dc2200640 1 -- 172.21.15.26:0/2893783814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dbc146f90 msgr2=0x7f1dbc15e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:01.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.718+0000 7f1dc2200640 1 --2- 172.21.15.26:0/2893783814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dbc146f90 0x7f1dbc15e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:01.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.718+0000 7f1dc2200640 1 -- 172.21.15.26:0/2893783814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1da4004d40 msgr2=0x7f1dbc1499a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:01.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.718+0000 7f1dc2200640 1 --2- 172.21.15.26:0/2893783814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1da4004d40 0x7f1dbc1499a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:01.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.718+0000 7f1dc2200640 1 -- 172.21.15.26:0/2893783814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dbc15eaf0 con 0x7f1dbc156280 2024-04-26T21:10:01.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.719+0000 7f1dc2200640 1 --2- 172.21.15.26:0/2893783814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dbc156280 0x7f1dbc15c0b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009ac0 tx=0x7f1db4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:01.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.719+0000 7f1db9600640 1 -- 172.21.15.26:0/2893783814 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4036040 con 0x7f1dbc156280 2024-04-26T21:10:01.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.719+0000 7f1db9600640 1 -- 172.21.15.26:0/2893783814 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db4041070 con 0x7f1dbc156280 2024-04-26T21:10:01.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.719+0000 7f1db9600640 1 -- 172.21.15.26:0/2893783814 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4009690 con 0x7f1dbc156280 2024-04-26T21:10:01.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.719+0000 7f1dc2c00640 1 -- 172.21.15.26:0/2893783814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dbc15a070 con 0x7f1dbc156280 2024-04-26T21:10:01.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.719+0000 7f1dc2c00640 1 -- 172.21.15.26:0/2893783814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dbc15a630 con 0x7f1dbc156280 2024-04-26T21:10:01.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.720+0000 7f1dc2c00640 1 -- 172.21.15.26:0/2893783814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dbc073c70 con 0x7f1dbc156280 2024-04-26T21:10:01.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.721+0000 7f1db9600640 1 -- 172.21.15.26:0/2893783814 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1db4039070 con 0x7f1dbc156280 2024-04-26T21:10:01.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.722+0000 7f1db9600640 1 --2- 172.21.15.26:0/2893783814 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1d9c06c8f0 0x7f1d9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:01.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.722+0000 7f1db9600640 1 -- 172.21.15.26:0/2893783814 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f1db40ab190 con 0x7f1dbc156280 2024-04-26T21:10:01.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.722+0000 7f1dc1800640 1 --2- 172.21.15.26:0/2893783814 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1d9c06c8f0 0x7f1d9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:01.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.723+0000 7f1dc1800640 1 --2- 172.21.15.26:0/2893783814 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1d9c06c8f0 0x7f1d9c06edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1db0001f80 tx=0x7f1db00012f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:01.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.726+0000 7f1db9600640 1 -- 172.21.15.26:0/2893783814 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1dbc073c70 con 0x7f1dbc156280 2024-04-26T21:10:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.972+0000 7f1dc2c00640 1 -- 172.21.15.26:0/2893783814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1dbc06a700 con 0x7f1dbc156280 2024-04-26T21:10:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.973+0000 7f1db9600640 1 -- 172.21.15.26:0/2893783814 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7f1db404a020 con 0x7f1dbc156280 2024-04-26T21:10:01.975 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:01.975 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2349160700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2349160700},{"type":"v1","addr":"172.21.15.26:6837","nonce":2349160700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4598,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2824191762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2824191762},{"type":"v1","addr":"172.21.15.86:6834","nonce":2824191762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/263746310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":263746310},{"type":"v1","addr":"172.21.15.86:6835","nonce":263746310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:09:48.622025+0000","modified":"2024-04-26T21:09:51.721103+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5058":{"gid":5058,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6836/2528180854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2528180854},{"type":"v1","addr":"172.21.15.26:6836","nonce":2528180854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5058,"qdb_cluster":[5058]},"id":2}]} 2024-04-26T21:10:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 17 2024-04-26T21:10:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.975+0000 7f1dc2c00640 1 -- 172.21.15.26:0/2893783814 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1d9c06c8f0 msgr2=0x7f1d9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.975+0000 7f1dc2c00640 1 --2- 172.21.15.26:0/2893783814 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1d9c06c8f0 0x7f1d9c06edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1db0001f80 tx=0x7f1db00012f0 comp rx=0 tx=0).stop 2024-04-26T21:10:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.975+0000 7f1dc2c00640 1 -- 172.21.15.26:0/2893783814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dbc156280 msgr2=0x7f1dbc15c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.975+0000 7f1dc2c00640 1 --2- 172.21.15.26:0/2893783814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dbc156280 0x7f1dbc15c0b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009ac0 tx=0x7f1db4004730 comp rx=0 tx=0).stop 2024-04-26T21:10:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.976+0000 7f1dc2c00640 1 -- 172.21.15.26:0/2893783814 shutdown_connections 2024-04-26T21:10:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.976+0000 7f1dc2c00640 1 --2- 172.21.15.26:0/2893783814 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f1d9c06c8f0 0x7f1d9c06edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.976+0000 7f1dc2c00640 1 --2- 172.21.15.26:0/2893783814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dbc156280 0x7f1dbc15c0b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.976+0000 7f1dc2c00640 1 --2- 172.21.15.26:0/2893783814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dbc146f90 0x7f1dbc15e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.976+0000 7f1dc2c00640 1 --2- 172.21.15.26:0/2893783814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1da4004d40 0x7f1dbc1499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.976+0000 7f1dc2c00640 1 -- 172.21.15.26:0/2893783814 >> 172.21.15.26:0/2893783814 conn(0x7f1dbc06c690 msgr2=0x7f1dbc0afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.976+0000 7f1dc2c00640 1 -- 172.21.15.26:0/2893783814 shutdown_connections 2024-04-26T21:10:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:01.976+0000 7f1dc2c00640 1 -- 172.21.15.26:0/2893783814 wait complete. 2024-04-26T21:10:01.996 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-26T21:10:02.236 INFO:teuthology.orchestra.run.smithi026.stdout:{"message":"","return_code":0} 2024-04-26T21:10:02.236 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-26T21:10:02.236 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001fb.00000000 parent 2024-04-26T21:10:02.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.341+0000 7f3d6cc5ef40 1 Processor -- start 2024-04-26T21:10:02.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.342+0000 7f3d6cc5ef40 1 -- start start 2024-04-26T21:10:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.342+0000 7f3d6cc5ef40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560f0bfb1f50 0x560f0bf96230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.342+0000 7f3d6cc5ef40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f0c03f050 0x560f0bf66940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.342+0000 7f3d6cc5ef40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f0c041eb0 0x560f0bfb3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.342+0000 7f3d6cc5ef40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560f0bfb2900 con 0x560f0c041eb0 2024-04-26T21:10:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.342+0000 7f3d6cc5ef40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560f0bf0ed20 con 0x560f0c03f050 2024-04-26T21:10:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.342+0000 7f3d6cc5ef40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x560f0c03f600 con 0x560f0bfb1f50 2024-04-26T21:10:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.343+0000 7f3d6a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f0c041eb0 0x560f0bfb3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.343+0000 7f3d6a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f0c041eb0 0x560f0bfb3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47456/0 (socket says 172.21.15.26:47456) 2024-04-26T21:10:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.343+0000 7f3d6a800640 1 -- 172.21.15.26:0/2779703294 learned_addr learned my addr 172.21.15.26:0/2779703294 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:02.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.343+0000 7f3d69e00640 1 --2- 172.21.15.26:0/2779703294 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560f0bfb1f50 0x560f0bf96230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:02.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.343+0000 7f3d61400640 1 --2- 172.21.15.26:0/2779703294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f0c03f050 0x560f0bf66940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:02.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.343+0000 7f3d6a800640 1 -- 172.21.15.26:0/2779703294 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560f0bfb1f50 msgr2=0x560f0bf96230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:02.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.343+0000 7f3d6a800640 1 --2- 172.21.15.26:0/2779703294 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560f0bfb1f50 0x560f0bf96230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.343+0000 7f3d6a800640 1 -- 172.21.15.26:0/2779703294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f0c03f050 msgr2=0x560f0bf66940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:02.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.343+0000 7f3d6a800640 1 --2- 172.21.15.26:0/2779703294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f0c03f050 0x560f0bf66940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.343+0000 7f3d6a800640 1 -- 172.21.15.26:0/2779703294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f0bfb5870 con 0x560f0c041eb0 2024-04-26T21:10:02.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.344+0000 7f3d6a800640 1 --2- 172.21.15.26:0/2779703294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f0c041eb0 0x560f0bfb3720 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3d6400a440 tx=0x7f3d6400a900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a739e4f6f84693 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:02.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.344+0000 7f3d68a00640 1 -- 172.21.15.26:0/2779703294 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6400b8d0 con 0x560f0c041eb0 2024-04-26T21:10:02.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.344+0000 7f3d68a00640 1 -- 172.21.15.26:0/2779703294 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d6400f030 con 0x560f0c041eb0 2024-04-26T21:10:02.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.344+0000 7f3d68a00640 1 -- 172.21.15.26:0/2779703294 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6400bc90 con 0x560f0c041eb0 2024-04-26T21:10:02.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.345+0000 7f3d6cc5ef40 1 -- 172.21.15.26:0/2779703294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f0c041eb0 msgr2=0x560f0bfb3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:02.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.345+0000 7f3d6cc5ef40 1 --2- 172.21.15.26:0/2779703294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f0c041eb0 0x560f0bfb3720 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3d6400a440 tx=0x7f3d6400a900 comp rx=0 tx=0).stop 2024-04-26T21:10:02.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.345+0000 7f3d6cc5ef40 1 -- 172.21.15.26:0/2779703294 shutdown_connections 2024-04-26T21:10:02.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.345+0000 7f3d6cc5ef40 1 --2- 172.21.15.26:0/2779703294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f0c041eb0 0x560f0bfb3720 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.345+0000 7f3d6cc5ef40 1 --2- 172.21.15.26:0/2779703294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f0c03f050 0x560f0bf66940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.345+0000 7f3d6cc5ef40 1 --2- 172.21.15.26:0/2779703294 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560f0bfb1f50 0x560f0bf96230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.345+0000 7f3d6cc5ef40 1 -- 172.21.15.26:0/2779703294 >> 172.21.15.26:0/2779703294 conn(0x560f0bf97100 msgr2=0x560f0bf651f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:02.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.345+0000 7f3d6cc5ef40 1 -- 172.21.15.26:0/2779703294 shutdown_connections 2024-04-26T21:10:02.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.345+0000 7f3d6cc5ef40 1 -- 172.21.15.26:0/2779703294 wait complete. 2024-04-26T21:10:02.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.346+0000 7f3d6cc5ef40 1 Processor -- start 2024-04-26T21:10:02.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.346+0000 7f3d6cc5ef40 1 -- start start 2024-04-26T21:10:02.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.347+0000 7f3d6cc5ef40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f0bfb1f50 0x560f0c0333e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:02.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.347+0000 7f3d6cc5ef40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560f0c03f050 0x560f0c030e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:02.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.347+0000 7f3d6cc5ef40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f0c041eb0 0x560f0bfb3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:02.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.347+0000 7f3d6cc5ef40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560f0bfb5b50 con 0x560f0bfb1f50 2024-04-26T21:10:02.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.347+0000 7f3d6cc5ef40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560f0bfb5ec0 con 0x560f0c041eb0 2024-04-26T21:10:02.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.347+0000 7f3d6cc5ef40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x560f0c035590 con 0x560f0c03f050 2024-04-26T21:10:02.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.347+0000 7f3d69e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f0bfb1f50 0x560f0c0333e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:02.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.347+0000 7f3d69e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f0bfb1f50 0x560f0c0333e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47462/0 (socket says 172.21.15.26:47462) 2024-04-26T21:10:02.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.347+0000 7f3d69e00640 1 -- 172.21.15.26:0/3346326123 learned_addr learned my addr 172.21.15.26:0/3346326123 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:02.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.347+0000 7f3d61400640 1 --2- 172.21.15.26:0/3346326123 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560f0c03f050 0x560f0c030e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:02.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.348+0000 7f3d6a800640 1 --2- 172.21.15.26:0/3346326123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f0c041eb0 0x560f0bfb3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:02.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.348+0000 7f3d69e00640 1 -- 172.21.15.26:0/3346326123 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560f0c03f050 msgr2=0x560f0c030e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:02.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.348+0000 7f3d69e00640 1 --2- 172.21.15.26:0/3346326123 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560f0c03f050 0x560f0c030e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.348+0000 7f3d69e00640 1 -- 172.21.15.26:0/3346326123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f0c041eb0 msgr2=0x560f0bfb3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:02.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.348+0000 7f3d69e00640 1 --2- 172.21.15.26:0/3346326123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f0c041eb0 0x560f0bfb3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.348+0000 7f3d69e00640 1 -- 172.21.15.26:0/3346326123 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f0c038c60 con 0x560f0bfb1f50 2024-04-26T21:10:02.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.348+0000 7f3d61400640 1 --2- 172.21.15.26:0/3346326123 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560f0c03f050 0x560f0c030e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:02.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.348+0000 7f3d6a800640 1 --2- 172.21.15.26:0/3346326123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560f0c041eb0 0x560f0bfb3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:02.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.348+0000 7f3d69e00640 1 --2- 172.21.15.26:0/3346326123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f0bfb1f50 0x560f0c0333e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00a410 tx=0x7f3d5c00a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:02.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.348+0000 7f3d63400640 1 -- 172.21.15.26:0/3346326123 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c010590 con 0x560f0bfb1f50 2024-04-26T21:10:02.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.348+0000 7f3d63400640 1 -- 172.21.15.26:0/3346326123 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d5c016070 con 0x560f0bfb1f50 2024-04-26T21:10:02.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.348+0000 7f3d63400640 1 -- 172.21.15.26:0/3346326123 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c010990 con 0x560f0bfb1f50 2024-04-26T21:10:02.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.348+0000 7f3d6cc5ef40 1 -- 172.21.15.26:0/3346326123 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560f0bf65390 con 0x560f0bfb1f50 2024-04-26T21:10:02.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.349+0000 7f3d6cc5ef40 1 -- 172.21.15.26:0/3346326123 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f0bf9aa00 con 0x560f0bfb1f50 2024-04-26T21:10:02.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.351+0000 7f3d63400640 1 -- 172.21.15.26:0/3346326123 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d5c010b30 con 0x560f0bfb1f50 2024-04-26T21:10:02.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.352+0000 7f3d63400640 1 --2- 172.21.15.26:0/3346326123 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3d4806b2d0 0x7f3d4806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:02.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.352+0000 7f3d63400640 1 -- 172.21.15.26:0/3346326123 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f3d5c08aa50 con 0x560f0bfb1f50 2024-04-26T21:10:02.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.352+0000 7f3d61400640 1 --2- 172.21.15.26:0/3346326123 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3d4806b2d0 0x7f3d4806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:02.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.352+0000 7f3d6cc5ef40 1 --2- 172.21.15.26:0/3346326123 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x560f0bfac120 0x560f0bfac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:02.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.352+0000 7f3d6cc5ef40 1 -- 172.21.15.26:0/3346326123 --> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] -- osd_op(unknown.0.0:1 5.13 5:ca8fdbba:::100000001fb.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) -- 0x560f0bfacb10 con 0x560f0bfac120 2024-04-26T21:10:02.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.352+0000 7f3d6a800640 1 --2- 172.21.15.26:0/3346326123 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x560f0bfac120 0x560f0bfac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:02.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.353+0000 7f3d61400640 1 --2- 172.21.15.26:0/3346326123 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3d4806b2d0 0x7f3d4806d790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3d54057000 tx=0x7f3d54052000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:02.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.353+0000 7f3d6a800640 1 --2- 172.21.15.26:0/3346326123 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x560f0bfac120 0x560f0bfac5d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:02.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.353+0000 7f3d6a800640 1 -- 172.21.15.26:0/3346326123 <== osd.3 v2:172.21.15.26:6809/1280758061 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f3d6400c7e0 con 0x560f0bfac120 2024-04-26T21:10:02.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.354+0000 7f3d6cc5ef40 1 -- 172.21.15.26:0/3346326123 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x560f0bfac120 msgr2=0x560f0bfac5d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:02.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.354+0000 7f3d6cc5ef40 1 --2- 172.21.15.26:0/3346326123 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x560f0bfac120 0x560f0bfac5d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.354+0000 7f3d6cc5ef40 1 -- 172.21.15.26:0/3346326123 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3d4806b2d0 msgr2=0x7f3d4806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:02.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.354+0000 7f3d6cc5ef40 1 --2- 172.21.15.26:0/3346326123 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3d4806b2d0 0x7f3d4806d790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3d54057000 tx=0x7f3d54052000 comp rx=0 tx=0).stop 2024-04-26T21:10:02.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.354+0000 7f3d6cc5ef40 1 -- 172.21.15.26:0/3346326123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f0bfb1f50 msgr2=0x560f0c0333e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:02.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.354+0000 7f3d6cc5ef40 1 --2- 172.21.15.26:0/3346326123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560f0bfb1f50 0x560f0c0333e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00a410 tx=0x7f3d5c00a8d0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.354+0000 7f3d6a800640 1 -- 172.21.15.26:0/3346326123 reap_dead start 2024-04-26T21:10:02.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.354+0000 7f3d6cc5ef40 1 -- 172.21.15.26:0/3346326123 shutdown_connections 2024-04-26T21:10:02.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.354+0000 7f3d6cc5ef40 1 -- 172.21.15.26:0/3346326123 >> 172.21.15.26:0/3346326123 conn(0x560f0bf97100 msgr2=0x560f0bf65fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:02.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.354+0000 7f3d6cc5ef40 1 -- 172.21.15.26:0/3346326123 shutdown_connections 2024-04-26T21:10:02.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.354+0000 7f3d6cc5ef40 1 -- 172.21.15.26:0/3346326123 wait complete. 2024-04-26T21:10:02.360 INFO:teuthology.orchestra.run.smithi026.stdout:Y��alphaparent_a 2024-04-26T21:10:02.360 DEBUG:teuthology.orchestra.run.smithi026:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-26T21:10:02.454 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-26T21:10:02.454 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628283, 2024-04-26T21:10:02.454 INFO:teuthology.orchestra.run.smithi026.stdout: "ancestors": [ 2024-04-26T21:10:02.454 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:10:02.454 INFO:teuthology.orchestra.run.smithi026.stdout: "dirino": 1099511628282, 2024-04-26T21:10:02.454 INFO:teuthology.orchestra.run.smithi026.stdout: "dname": "alpha", 2024-04-26T21:10:02.454 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 2 2024-04-26T21:10:02.454 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:10:02.454 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:10:02.454 INFO:teuthology.orchestra.run.smithi026.stdout: "dirino": 1, 2024-04-26T21:10:02.455 INFO:teuthology.orchestra.run.smithi026.stdout: "dname": "parent_a", 2024-04-26T21:10:02.455 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 18 2024-04-26T21:10:02.455 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:10:02.455 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-26T21:10:02.455 INFO:teuthology.orchestra.run.smithi026.stdout: "pool": 5, 2024-04-26T21:10:02.455 INFO:teuthology.orchestra.run.smithi026.stdout: "old_pools": [] 2024-04-26T21:10:02.455 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-26T21:10:02.455 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:02.461 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001fb.00000000 layout 2024-04-26T21:10:02.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.575+0000 7f8e643d2f40 1 Processor -- start 2024-04-26T21:10:02.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.575+0000 7f8e643d2f40 1 -- start start 2024-04-26T21:10:02.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.576+0000 7f8e643d2f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbed239ed0 0x55bbed30a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:02.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.576+0000 7f8e643d2f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bbed23a570 0x55bbed266610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:02.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.576+0000 7f8e643d2f40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bbed30c7c0 0x55bbed266b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:02.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.576+0000 7f8e643d2f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bbed1ded20 con 0x55bbed23a570 2024-04-26T21:10:02.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.576+0000 7f8e643d2f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bbed1e2d60 con 0x55bbed239ed0 2024-04-26T21:10:02.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.576+0000 7f8e643d2f40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55bbed265ea0 con 0x55bbed30c7c0 2024-04-26T21:10:02.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.576+0000 7f8e58c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bbed23a570 0x55bbed266610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:02.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.576+0000 7f8e58c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bbed23a570 0x55bbed266610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47480/0 (socket says 172.21.15.26:47480) 2024-04-26T21:10:02.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.576+0000 7f8e58c00640 1 -- 172.21.15.26:0/1395488572 learned_addr learned my addr 172.21.15.26:0/1395488572 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:02.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.577+0000 7f8e62000640 1 --2- 172.21.15.26:0/1395488572 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bbed30c7c0 0x55bbed266b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:02.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.577+0000 7f8e61600640 1 --2- 172.21.15.26:0/1395488572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbed239ed0 0x55bbed30a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:02.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.577+0000 7f8e58c00640 1 -- 172.21.15.26:0/1395488572 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bbed30c7c0 msgr2=0x55bbed266b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:02.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.577+0000 7f8e58c00640 1 --2- 172.21.15.26:0/1395488572 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bbed30c7c0 0x55bbed266b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.577+0000 7f8e58c00640 1 -- 172.21.15.26:0/1395488572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbed239ed0 msgr2=0x55bbed30a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:02.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.577+0000 7f8e58c00640 1 --2- 172.21.15.26:0/1395488572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbed239ed0 0x55bbed30a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.577+0000 7f8e58c00640 1 -- 172.21.15.26:0/1395488572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbed281be0 con 0x55bbed23a570 2024-04-26T21:10:02.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.577+0000 7f8e58c00640 1 --2- 172.21.15.26:0/1395488572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bbed23a570 0x55bbed266610 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c002de0 tx=0x7f8e4c00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6d0fa2f28a5ea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:02.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.578+0000 7f8e5be00640 1 -- 172.21.15.26:0/1395488572 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c00f400 con 0x55bbed23a570 2024-04-26T21:10:02.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.578+0000 7f8e5be00640 1 -- 172.21.15.26:0/1395488572 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e4c014070 con 0x55bbed23a570 2024-04-26T21:10:02.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.578+0000 7f8e5be00640 1 -- 172.21.15.26:0/1395488572 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c00f7e0 con 0x55bbed23a570 2024-04-26T21:10:02.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.578+0000 7f8e643d2f40 1 -- 172.21.15.26:0/1395488572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bbed23a570 msgr2=0x55bbed266610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:02.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.578+0000 7f8e643d2f40 1 --2- 172.21.15.26:0/1395488572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bbed23a570 0x55bbed266610 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c002de0 tx=0x7f8e4c00b5b0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.578+0000 7f8e643d2f40 1 -- 172.21.15.26:0/1395488572 shutdown_connections 2024-04-26T21:10:02.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.578+0000 7f8e643d2f40 1 --2- 172.21.15.26:0/1395488572 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bbed30c7c0 0x55bbed266b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.578+0000 7f8e643d2f40 1 --2- 172.21.15.26:0/1395488572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bbed23a570 0x55bbed266610 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.578+0000 7f8e643d2f40 1 --2- 172.21.15.26:0/1395488572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbed239ed0 0x55bbed30a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.578+0000 7f8e643d2f40 1 -- 172.21.15.26:0/1395488572 >> 172.21.15.26:0/1395488572 conn(0x55bbed264ba0 msgr2=0x55bbed281430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:02.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.578+0000 7f8e643d2f40 1 -- 172.21.15.26:0/1395488572 shutdown_connections 2024-04-26T21:10:02.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.578+0000 7f8e643d2f40 1 -- 172.21.15.26:0/1395488572 wait complete. 2024-04-26T21:10:02.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.579+0000 7f8e643d2f40 1 Processor -- start 2024-04-26T21:10:02.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.579+0000 7f8e643d2f40 1 -- start start 2024-04-26T21:10:02.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.580+0000 7f8e643d2f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bbed239ed0 0x55bbed3061f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:02.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.580+0000 7f8e643d2f40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bbed23a570 0x55bbed3014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:02.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.580+0000 7f8e643d2f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbed30c7c0 0x55bbed2ff7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:02.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.580+0000 7f8e643d2f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bbed30bb80 con 0x55bbed239ed0 2024-04-26T21:10:02.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.580+0000 7f8e643d2f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bbed30b930 con 0x55bbed30c7c0 2024-04-26T21:10:02.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.580+0000 7f8e643d2f40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55bbed3029f0 con 0x55bbed23a570 2024-04-26T21:10:02.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.581+0000 7f8e61600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bbed239ed0 0x55bbed3061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:02.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.581+0000 7f8e61600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bbed239ed0 0x55bbed3061f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47490/0 (socket says 172.21.15.26:47490) 2024-04-26T21:10:02.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.581+0000 7f8e61600640 1 -- 172.21.15.26:0/1667112118 learned_addr learned my addr 172.21.15.26:0/1667112118 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:02.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.581+0000 7f8e58c00640 1 --2- 172.21.15.26:0/1667112118 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bbed23a570 0x55bbed3014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:02.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.581+0000 7f8e62000640 1 --2- 172.21.15.26:0/1667112118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbed30c7c0 0x55bbed2ff7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:02.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.581+0000 7f8e61600640 1 -- 172.21.15.26:0/1667112118 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bbed23a570 msgr2=0x55bbed3014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:02.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.581+0000 7f8e61600640 1 --2- 172.21.15.26:0/1667112118 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55bbed23a570 0x55bbed3014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.581+0000 7f8e61600640 1 -- 172.21.15.26:0/1667112118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbed30c7c0 msgr2=0x55bbed2ff7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:02.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.581+0000 7f8e61600640 1 --2- 172.21.15.26:0/1667112118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bbed30c7c0 0x55bbed2ff7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.581+0000 7f8e61600640 1 -- 172.21.15.26:0/1667112118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbed304390 con 0x55bbed239ed0 2024-04-26T21:10:02.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.581+0000 7f8e61600640 1 --2- 172.21.15.26:0/1667112118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bbed239ed0 0x55bbed3061f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8e54006f60 tx=0x7f8e54006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:02.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.582+0000 7f8e5aa00640 1 -- 172.21.15.26:0/1667112118 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54017070 con 0x55bbed239ed0 2024-04-26T21:10:02.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.582+0000 7f8e5aa00640 1 -- 172.21.15.26:0/1667112118 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e540062d0 con 0x55bbed239ed0 2024-04-26T21:10:02.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.582+0000 7f8e5aa00640 1 -- 172.21.15.26:0/1667112118 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e540046e0 con 0x55bbed239ed0 2024-04-26T21:10:02.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.582+0000 7f8e643d2f40 1 -- 172.21.15.26:0/1667112118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bbed301c60 con 0x55bbed239ed0 2024-04-26T21:10:02.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.582+0000 7f8e643d2f40 1 -- 172.21.15.26:0/1667112118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbed301f40 con 0x55bbed239ed0 2024-04-26T21:10:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.584+0000 7f8e5aa00640 1 -- 172.21.15.26:0/1667112118 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8e5401a070 con 0x55bbed239ed0 2024-04-26T21:10:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.585+0000 7f8e5aa00640 1 --2- 172.21.15.26:0/1667112118 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8e4006b320 0x7f8e4006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.585+0000 7f8e5aa00640 1 -- 172.21.15.26:0/1667112118 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f8e54089f00 con 0x55bbed239ed0 2024-04-26T21:10:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.585+0000 7f8e58c00640 1 --2- 172.21.15.26:0/1667112118 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8e4006b320 0x7f8e4006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.585+0000 7f8e643d2f40 1 --2- 172.21.15.26:0/1667112118 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x55bbed2fdc40 0x55bbed3ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.585+0000 7f8e643d2f40 1 -- 172.21.15.26:0/1667112118 --> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] -- osd_op(unknown.0.0:1 5.13 5:ca8fdbba:::100000001fb.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) -- 0x55bbed3ae970 con 0x55bbed2fdc40 2024-04-26T21:10:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.586+0000 7f8e62000640 1 --2- 172.21.15.26:0/1667112118 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x55bbed2fdc40 0x55bbed3ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.586+0000 7f8e58c00640 1 --2- 172.21.15.26:0/1667112118 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8e4006b320 0x7f8e4006d7e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55bbed280af0 tx=0x7f8e4c008000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.587+0000 7f8e62000640 1 --2- 172.21.15.26:0/1667112118 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x55bbed2fdc40 0x55bbed3ae430 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.587+0000 7f8e62000640 1 -- 172.21.15.26:0/1667112118 <== osd.3 v2:172.21.15.26:6809/1280758061 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (30) out=30b] v0'0 uv1 ondisk = 0) ==== 164+0+30 (crc 0 0 0) 0x7f8e5c054570 con 0x55bbed2fdc40 2024-04-26T21:10:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.587+0000 7f8e643d2f40 1 -- 172.21.15.26:0/1667112118 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x55bbed2fdc40 msgr2=0x55bbed3ae430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.587+0000 7f8e643d2f40 1 --2- 172.21.15.26:0/1667112118 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x55bbed2fdc40 0x55bbed3ae430 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.587+0000 7f8e643d2f40 1 -- 172.21.15.26:0/1667112118 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8e4006b320 msgr2=0x7f8e4006d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.587+0000 7f8e643d2f40 1 --2- 172.21.15.26:0/1667112118 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8e4006b320 0x7f8e4006d7e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55bbed280af0 tx=0x7f8e4c008000 comp rx=0 tx=0).stop 2024-04-26T21:10:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.587+0000 7f8e643d2f40 1 -- 172.21.15.26:0/1667112118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bbed239ed0 msgr2=0x55bbed3061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.587+0000 7f8e643d2f40 1 --2- 172.21.15.26:0/1667112118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bbed239ed0 0x55bbed3061f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8e54006f60 tx=0x7f8e54006f90 comp rx=0 tx=0).stop 2024-04-26T21:10:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.587+0000 7f8e62000640 1 -- 172.21.15.26:0/1667112118 reap_dead start 2024-04-26T21:10:02.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.587+0000 7f8e643d2f40 1 -- 172.21.15.26:0/1667112118 shutdown_connections 2024-04-26T21:10:02.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.587+0000 7f8e643d2f40 1 -- 172.21.15.26:0/1667112118 >> 172.21.15.26:0/1667112118 conn(0x55bbed264ba0 msgr2=0x55bbed2fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:02.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.587+0000 7f8e643d2f40 1 -- 172.21.15.26:0/1667112118 shutdown_connections 2024-04-26T21:10:02.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:02.587+0000 7f8e643d2f40 1 -- 172.21.15.26:0/1667112118 wait complete. 2024-04-26T21:10:02.593 INFO:teuthology.orchestra.run.smithi026.stdout:@@ 2024-04-26T21:10:02.594 DEBUG:teuthology.orchestra.run.smithi026:> ceph-dencoder type file_layout_t import - decode dump_json 2024-04-26T21:10:02.689 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-26T21:10:02.689 INFO:teuthology.orchestra.run.smithi026.stdout: "stripe_unit": 4194304, 2024-04-26T21:10:02.689 INFO:teuthology.orchestra.run.smithi026.stdout: "stripe_count": 1, 2024-04-26T21:10:02.689 INFO:teuthology.orchestra.run.smithi026.stdout: "object_size": 4194304, 2024-04-26T21:10:02.689 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_id": 5, 2024-04-26T21:10:02.689 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_ns": "" 2024-04-26T21:10:02.689 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-26T21:10:02.690 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:02.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:10:02.695 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_b) 2024-04-26T21:10:02.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:10:02.750 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mv parent_a/alpha parent_b/alpha) 2024-04-26T21:10:02.811 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:10:03.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.012+0000 7f60b2200640 1 Processor -- start 2024-04-26T21:10:03.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.012+0000 7f60b2200640 1 -- start start 2024-04-26T21:10:03.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.014+0000 7f60b2200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f60ac0d06f0 0x7f60ac0d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:03.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.014+0000 7f60b2200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60ac0cf690 con 0x7f60ac0d1670 2024-04-26T21:10:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.014+0000 7f60b2200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60ac0cf930 con 0x7f60ac145c20 2024-04-26T21:10:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.014+0000 7f60b2200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f60ac0ced10 con 0x7f60ac0d06f0 2024-04-26T21:10:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.014+0000 7f60b1800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f60ac0d1670 0x7f60ac071fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55458/0 (socket says 172.21.15.26:55458) 2024-04-26T21:10:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.014+0000 7f60b1800640 1 -- 172.21.15.26:0/2199685792 learned_addr learned my addr 172.21.15.26:0/2199685792 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.015+0000 7f60a3e00640 1 --2- 172.21.15.26:0/2199685792 >> v2:172.21.15.86:3300/0 conn(0x7f60ac0d06f0 0x7f60ac0d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.015+0000 7f60abe00640 1 -- 172.21.15.26:0/2199685792 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 101600040 0 0) 0x7f60ac0cf690 con 0x7f60ac0d1670 2024-04-26T21:10:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.015+0000 7f60abe00640 1 -- 172.21.15.26:0/2199685792 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6090003520 con 0x7f60ac0d1670 2024-04-26T21:10:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.015+0000 7f60a3e00640 1 -- 172.21.15.26:0/2199685792 >> v1:172.21.15.86:6789/0 conn(0x7f60ac145c20 legacy=0x7f60ac146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.016+0000 7f60a3e00640 1 -- 172.21.15.26:0/2199685792 >> v1:172.21.15.26:6789/0 conn(0x7f60ac0d1670 legacy=0x7f60ac071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.016+0000 7f60abe00640 1 -- 172.21.15.26:0/2199685792 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3363160567 0 0) 0x7f6090003520 con 0x7f60ac0d1670 2024-04-26T21:10:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.016+0000 7f60a3e00640 1 -- 172.21.15.26:0/2199685792 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ac0cfc10 con 0x7f60ac0d06f0 2024-04-26T21:10:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.017+0000 7f60a3e00640 1 --2- 172.21.15.26:0/2199685792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60ac0d06f0 0x7f60ac0d1130 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f60940097a0 tx=0x7f609402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ee12c8746e0e90e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:03.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.017+0000 7f60abe00640 1 -- 172.21.15.26:0/2199685792 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094037070 con 0x7f60ac0d06f0 2024-04-26T21:10:03.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.017+0000 7f60abe00640 1 -- 172.21.15.26:0/2199685792 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f609402ac50 con 0x7f60ac0d06f0 2024-04-26T21:10:03.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.017+0000 7f60abe00640 1 -- 172.21.15.26:0/2199685792 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4001a10 con 0x7f60ac0d06f0 2024-04-26T21:10:03.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.018+0000 7f60b2200640 1 -- 172.21.15.26:0/2199685792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60ac0d06f0 msgr2=0x7f60ac0d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:03.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.018+0000 7f60b2200640 1 --2- 172.21.15.26:0/2199685792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60ac0d06f0 0x7f60ac0d1130 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f60940097a0 tx=0x7f609402a1c0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.018+0000 7f60b2200640 1 -- 172.21.15.26:0/2199685792 shutdown_connections 2024-04-26T21:10:03.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.018+0000 7f60b2200640 1 --2- 172.21.15.26:0/2199685792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60ac0d06f0 0x7f60ac0d1130 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.018+0000 7f60b2200640 1 -- 172.21.15.26:0/2199685792 >> 172.21.15.26:0/2199685792 conn(0x7f60ac06c690 msgr2=0x7f60ac06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:03.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.018+0000 7f60b2200640 1 -- 172.21.15.26:0/2199685792 shutdown_connections 2024-04-26T21:10:03.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.018+0000 7f60b2200640 1 -- 172.21.15.26:0/2199685792 wait complete. 2024-04-26T21:10:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.021+0000 7f60b2200640 1 Processor -- start 2024-04-26T21:10:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.021+0000 7f60b2200640 1 -- start start 2024-04-26T21:10:03.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.022+0000 7f60b2200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60ac0d06f0 0x7f60ac1e7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:03.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.022+0000 7f60b2200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60ac0d1670 0x7f60ac155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:03.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.022+0000 7f60b2200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60ac145c20 0x7f60ac15f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:03.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.022+0000 7f60b2200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac0d1aa0 con 0x7f60ac0d1670 2024-04-26T21:10:03.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.022+0000 7f60b2200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac0d01d0 con 0x7f60ac0d06f0 2024-04-26T21:10:03.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.022+0000 7f60b2200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f60ac146b60 con 0x7f60ac145c20 2024-04-26T21:10:03.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.023+0000 7f60a3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60ac0d1670 0x7f60ac155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:03.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.023+0000 7f60a3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60ac0d1670 0x7f60ac155a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47492/0 (socket says 172.21.15.26:47492) 2024-04-26T21:10:03.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.023+0000 7f60a3e00640 1 -- 172.21.15.26:0/2176522640 learned_addr learned my addr 172.21.15.26:0/2176522640 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:03.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.023+0000 7f60b0e00640 1 --2- 172.21.15.26:0/2176522640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60ac0d06f0 0x7f60ac1e7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:03.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.023+0000 7f60b1800640 1 --2- 172.21.15.26:0/2176522640 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60ac145c20 0x7f60ac15f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:03.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.023+0000 7f60a3e00640 1 -- 172.21.15.26:0/2176522640 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60ac145c20 msgr2=0x7f60ac15f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:03.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.023+0000 7f60a3e00640 1 --2- 172.21.15.26:0/2176522640 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60ac145c20 0x7f60ac15f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.023+0000 7f60a3e00640 1 -- 172.21.15.26:0/2176522640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60ac0d06f0 msgr2=0x7f60ac1e7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:03.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.023+0000 7f60a3e00640 1 --2- 172.21.15.26:0/2176522640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60ac0d06f0 0x7f60ac1e7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.023+0000 7f60a3e00640 1 -- 172.21.15.26:0/2176522640 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ac156210 con 0x7f60ac0d1670 2024-04-26T21:10:03.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.024+0000 7f60a3e00640 1 --2- 172.21.15.26:0/2176522640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60ac0d1670 0x7f60ac155a80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f60940098c0 tx=0x7f60940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:03.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.024+0000 7f60a9600640 1 -- 172.21.15.26:0/2176522640 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094037040 con 0x7f60ac0d1670 2024-04-26T21:10:03.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.024+0000 7f60a9600640 1 -- 172.21.15.26:0/2176522640 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6094004030 con 0x7f60ac0d1670 2024-04-26T21:10:03.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.024+0000 7f60a9600640 1 -- 172.21.15.26:0/2176522640 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094004400 con 0x7f60ac0d1670 2024-04-26T21:10:03.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.024+0000 7f60b2200640 1 -- 172.21.15.26:0/2176522640 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ac1e77b0 con 0x7f60ac0d1670 2024-04-26T21:10:03.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.024+0000 7f60b2200640 1 -- 172.21.15.26:0/2176522640 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ac156790 con 0x7f60ac0d1670 2024-04-26T21:10:03.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.025+0000 7f60b2200640 1 -- 172.21.15.26:0/2176522640 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60ac06a700 con 0x7f60ac0d1670 2024-04-26T21:10:03.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.026+0000 7f60a9600640 1 -- 172.21.15.26:0/2176522640 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6094003610 con 0x7f60ac0d1670 2024-04-26T21:10:03.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.027+0000 7f60a9600640 1 --2- 172.21.15.26:0/2176522640 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f608806c920 0x7f608806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:03.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.027+0000 7f60a9600640 1 -- 172.21.15.26:0/2176522640 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f60940ac7f0 con 0x7f60ac0d1670 2024-04-26T21:10:03.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.027+0000 7f60b0e00640 1 --2- 172.21.15.26:0/2176522640 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f608806c920 0x7f608806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:03.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.028+0000 7f60b0e00640 1 --2- 172.21.15.26:0/2176522640 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f608806c920 0x7f608806ede0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f609c007fd0 tx=0x7f609c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:03.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.032+0000 7f60a9600640 1 -- 172.21.15.26:0/2176522640 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f609407ad30 con 0x7f60ac0d1670 2024-04-26T21:10:03.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.278+0000 7f60b2200640 1 -- 172.21.15.26:0/2176522640 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60ac155fc0 con 0x7f60ac0d1670 2024-04-26T21:10:03.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.279+0000 7f60a9600640 1 -- 172.21.15.26:0/2176522640 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7f609407a630 con 0x7f60ac0d1670 2024-04-26T21:10:03.281 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:03.281 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2349160700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2349160700},{"type":"v1","addr":"172.21.15.26:6837","nonce":2349160700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4598,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2824191762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2824191762},{"type":"v1","addr":"172.21.15.86:6834","nonce":2824191762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/263746310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":263746310},{"type":"v1","addr":"172.21.15.86:6835","nonce":263746310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:09:48.622025+0000","modified":"2024-04-26T21:09:51.721103+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5058":{"gid":5058,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6836/2528180854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2528180854},{"type":"v1","addr":"172.21.15.26:6836","nonce":2528180854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5058,"qdb_cluster":[5058]},"id":2}]} 2024-04-26T21:10:03.281 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 17 2024-04-26T21:10:03.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.282+0000 7f60b2200640 1 -- 172.21.15.26:0/2176522640 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f608806c920 msgr2=0x7f608806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:03.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.282+0000 7f60b2200640 1 --2- 172.21.15.26:0/2176522640 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f608806c920 0x7f608806ede0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f609c007fd0 tx=0x7f609c009040 comp rx=0 tx=0).stop 2024-04-26T21:10:03.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.282+0000 7f60b2200640 1 -- 172.21.15.26:0/2176522640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60ac0d1670 msgr2=0x7f60ac155a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:03.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.282+0000 7f60b2200640 1 --2- 172.21.15.26:0/2176522640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60ac0d1670 0x7f60ac155a80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f60940098c0 tx=0x7f60940046c0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.283+0000 7f60b2200640 1 -- 172.21.15.26:0/2176522640 shutdown_connections 2024-04-26T21:10:03.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.283+0000 7f60b2200640 1 --2- 172.21.15.26:0/2176522640 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f608806c920 0x7f608806ede0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.283+0000 7f60b2200640 1 --2- 172.21.15.26:0/2176522640 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60ac145c20 0x7f60ac15f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.283+0000 7f60b2200640 1 --2- 172.21.15.26:0/2176522640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60ac0d1670 0x7f60ac155a80 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.283+0000 7f60b2200640 1 --2- 172.21.15.26:0/2176522640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60ac0d06f0 0x7f60ac1e7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.283+0000 7f60b2200640 1 -- 172.21.15.26:0/2176522640 >> 172.21.15.26:0/2176522640 conn(0x7f60ac06c690 msgr2=0x7f60ac0b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:03.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.283+0000 7f60b2200640 1 -- 172.21.15.26:0/2176522640 shutdown_connections 2024-04-26T21:10:03.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.283+0000 7f60b2200640 1 -- 172.21.15.26:0/2176522640 wait complete. 2024-04-26T21:10:03.304 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-26T21:10:03.557 INFO:teuthology.orchestra.run.smithi026.stdout:{"message":"","return_code":0} 2024-04-26T21:10:03.557 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-26T21:10:03.557 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001fb.00000000 parent 2024-04-26T21:10:03.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.656+0000 7f05c971df40 1 Processor -- start 2024-04-26T21:10:03.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.656+0000 7f05c971df40 1 -- start start 2024-04-26T21:10:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.657+0000 7f05c971df40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c1faf50 0x55ac0c1ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.657+0000 7f05c971df40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ac0c288050 0x55ac0c289420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.657+0000 7f05c971df40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac0c28aeb0 0x55ac0c1affd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.657+0000 7f05c971df40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ac0c288600 con 0x55ac0c28aeb0 2024-04-26T21:10:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.657+0000 7f05c971df40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ac0c1fde70 con 0x55ac0c1faf50 2024-04-26T21:10:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.657+0000 7f05c971df40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55ac0c288b80 con 0x55ac0c288050 2024-04-26T21:10:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.657+0000 7f05c6800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac0c28aeb0 0x55ac0c1affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.657+0000 7f05c6800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac0c28aeb0 0x55ac0c1affd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47516/0 (socket says 172.21.15.26:47516) 2024-04-26T21:10:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.657+0000 7f05c6800640 1 -- 172.21.15.26:0/4212191749 learned_addr learned my addr 172.21.15.26:0/4212191749 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:03.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.658+0000 7f05c5400640 1 --2- 172.21.15.26:0/4212191749 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ac0c288050 0x55ac0c289420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:03.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.658+0000 7f05c5e00640 1 --2- 172.21.15.26:0/4212191749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c1faf50 0x55ac0c1ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:03.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.658+0000 7f05c6800640 1 -- 172.21.15.26:0/4212191749 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ac0c288050 msgr2=0x55ac0c289420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:03.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.658+0000 7f05c6800640 1 --2- 172.21.15.26:0/4212191749 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ac0c288050 0x55ac0c289420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.658+0000 7f05c6800640 1 -- 172.21.15.26:0/4212191749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c1faf50 msgr2=0x55ac0c1ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:03.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.658+0000 7f05c6800640 1 --2- 172.21.15.26:0/4212191749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c1faf50 0x55ac0c1ae7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.658+0000 7f05c6800640 1 -- 172.21.15.26:0/4212191749 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac0c0dab30 con 0x55ac0c28aeb0 2024-04-26T21:10:03.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.658+0000 7f05c6800640 1 --2- 172.21.15.26:0/4212191749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac0c28aeb0 0x55ac0c1affd0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f05c0025ca0 tx=0x7f05c0051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57c0ccb506a4a9d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:03.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.658+0000 7f05bfe00640 1 -- 172.21.15.26:0/4212191749 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0062070 con 0x55ac0c28aeb0 2024-04-26T21:10:03.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.658+0000 7f05bfe00640 1 -- 172.21.15.26:0/4212191749 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c0054970 con 0x55ac0c28aeb0 2024-04-26T21:10:03.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.658+0000 7f05bfe00640 1 -- 172.21.15.26:0/4212191749 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0054d20 con 0x55ac0c28aeb0 2024-04-26T21:10:03.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.659+0000 7f05c971df40 1 -- 172.21.15.26:0/4212191749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac0c28aeb0 msgr2=0x55ac0c1affd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:03.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.659+0000 7f05c971df40 1 --2- 172.21.15.26:0/4212191749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac0c28aeb0 0x55ac0c1affd0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f05c0025ca0 tx=0x7f05c0051000 comp rx=0 tx=0).stop 2024-04-26T21:10:03.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.659+0000 7f05c971df40 1 -- 172.21.15.26:0/4212191749 shutdown_connections 2024-04-26T21:10:03.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.659+0000 7f05c971df40 1 --2- 172.21.15.26:0/4212191749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac0c28aeb0 0x55ac0c1affd0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.659+0000 7f05c971df40 1 --2- 172.21.15.26:0/4212191749 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ac0c288050 0x55ac0c289420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.659+0000 7f05c971df40 1 --2- 172.21.15.26:0/4212191749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c1faf50 0x55ac0c1ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.659+0000 7f05c971df40 1 -- 172.21.15.26:0/4212191749 >> 172.21.15.26:0/4212191749 conn(0x55ac0c1e0100 msgr2=0x55ac0c289fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:03.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.659+0000 7f05c971df40 1 -- 172.21.15.26:0/4212191749 shutdown_connections 2024-04-26T21:10:03.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.659+0000 7f05c971df40 1 -- 172.21.15.26:0/4212191749 wait complete. 2024-04-26T21:10:03.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.660+0000 7f05c971df40 1 Processor -- start 2024-04-26T21:10:03.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.660+0000 7f05c971df40 1 -- start start 2024-04-26T21:10:03.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.661+0000 7f05c971df40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c1faf50 0x55ac0c1fc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:03.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.661+0000 7f05c971df40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac0c288050 0x55ac0c1e2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:03.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.661+0000 7f05c971df40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ac0c28aeb0 0x55ac0c1e3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:03.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.661+0000 7f05c971df40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ac0c2821b0 con 0x55ac0c288050 2024-04-26T21:10:03.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.661+0000 7f05c971df40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ac0c27ae00 con 0x55ac0c1faf50 2024-04-26T21:10:03.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.661+0000 7f05c971df40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55ac0c1df210 con 0x55ac0c28aeb0 2024-04-26T21:10:03.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.661+0000 7f05c5400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac0c288050 0x55ac0c1e2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.661+0000 7f05c5400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac0c288050 0x55ac0c1e2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47530/0 (socket says 172.21.15.26:47530) 2024-04-26T21:10:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.661+0000 7f05c5400640 1 -- 172.21.15.26:0/1300436540 learned_addr learned my addr 172.21.15.26:0/1300436540 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.662+0000 7f05c5400640 1 -- 172.21.15.26:0/1300436540 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ac0c28aeb0 msgr2=0x55ac0c1e3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.662+0000 7f05c6800640 1 --2- 172.21.15.26:0/1300436540 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ac0c28aeb0 0x55ac0c1e3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.662+0000 7f05c5e00640 1 --2- 172.21.15.26:0/1300436540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c1faf50 0x55ac0c1fc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.662+0000 7f05c5400640 1 --2- 172.21.15.26:0/1300436540 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55ac0c28aeb0 0x55ac0c1e3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.662+0000 7f05c5400640 1 -- 172.21.15.26:0/1300436540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c1faf50 msgr2=0x55ac0c1fc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.662+0000 7f05c5400640 1 --2- 172.21.15.26:0/1300436540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac0c1faf50 0x55ac0c1fc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.662+0000 7f05c5400640 1 -- 172.21.15.26:0/1300436540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac0c289790 con 0x55ac0c288050 2024-04-26T21:10:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.662+0000 7f05c5400640 1 --2- 172.21.15.26:0/1300436540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac0c288050 0x55ac0c1e2dc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f05b000bdf0 tx=0x7f05b000bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.662+0000 7f05bea00640 1 -- 172.21.15.26:0/1300436540 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0007b50 con 0x55ac0c288050 2024-04-26T21:10:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.662+0000 7f05bea00640 1 -- 172.21.15.26:0/1300436540 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b0007cf0 con 0x55ac0c288050 2024-04-26T21:10:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.662+0000 7f05c971df40 1 -- 172.21.15.26:0/1300436540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ac0c289a70 con 0x55ac0c288050 2024-04-26T21:10:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.662+0000 7f05bea00640 1 -- 172.21.15.26:0/1300436540 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0015670 con 0x55ac0c288050 2024-04-26T21:10:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.663+0000 7f05c971df40 1 -- 172.21.15.26:0/1300436540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac0c1f75d0 con 0x55ac0c288050 2024-04-26T21:10:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.665+0000 7f05bea00640 1 -- 172.21.15.26:0/1300436540 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f05b0009410 con 0x55ac0c288050 2024-04-26T21:10:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.665+0000 7f05bea00640 1 --2- 172.21.15.26:0/1300436540 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05a406b2d0 0x7f05a406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.666+0000 7f05bea00640 1 -- 172.21.15.26:0/1300436540 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f05b0089e00 con 0x55ac0c288050 2024-04-26T21:10:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.666+0000 7f05c5e00640 1 --2- 172.21.15.26:0/1300436540 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05a406b2d0 0x7f05a406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.666+0000 7f05c971df40 1 --2- 172.21.15.26:0/1300436540 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x55ac0c1f7dd0 0x55ac0c329960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.666+0000 7f05c971df40 1 -- 172.21.15.26:0/1300436540 --> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] -- osd_op(unknown.0.0:1 5.13 5:ca8fdbba:::100000001fb.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34) -- 0x55ac0c1fcab0 con 0x55ac0c1f7dd0 2024-04-26T21:10:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.666+0000 7f05c6800640 1 --2- 172.21.15.26:0/1300436540 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x55ac0c1f7dd0 0x55ac0c329960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.667+0000 7f05c5e00640 1 --2- 172.21.15.26:0/1300436540 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05a406b2d0 0x7f05a406d790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55ac0c28abc0 tx=0x7f05b8007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.667+0000 7f05c6800640 1 --2- 172.21.15.26:0/1300436540 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x55ac0c1f7dd0 0x55ac0c329960 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.667+0000 7f05c6800640 1 -- 172.21.15.26:0/1300436540 <== osd.3 v2:172.21.15.26:6809/1280758061 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (95) out=95b] v0'0 uv2 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f05c0064030 con 0x55ac0c1f7dd0 2024-04-26T21:10:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.667+0000 7f05c971df40 1 -- 172.21.15.26:0/1300436540 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x55ac0c1f7dd0 msgr2=0x55ac0c329960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.667+0000 7f05c971df40 1 --2- 172.21.15.26:0/1300436540 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x55ac0c1f7dd0 0x55ac0c329960 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.667+0000 7f05c971df40 1 -- 172.21.15.26:0/1300436540 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05a406b2d0 msgr2=0x7f05a406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.667+0000 7f05c971df40 1 --2- 172.21.15.26:0/1300436540 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f05a406b2d0 0x7f05a406d790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55ac0c28abc0 tx=0x7f05b8007040 comp rx=0 tx=0).stop 2024-04-26T21:10:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.667+0000 7f05c971df40 1 -- 172.21.15.26:0/1300436540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac0c288050 msgr2=0x55ac0c1e2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.667+0000 7f05c971df40 1 --2- 172.21.15.26:0/1300436540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ac0c288050 0x55ac0c1e2dc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f05b000bdf0 tx=0x7f05b000bee0 comp rx=0 tx=0).stop 2024-04-26T21:10:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.667+0000 7f05c6800640 1 -- 172.21.15.26:0/1300436540 reap_dead start 2024-04-26T21:10:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.668+0000 7f05c971df40 1 -- 172.21.15.26:0/1300436540 shutdown_connections 2024-04-26T21:10:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.668+0000 7f05c971df40 1 -- 172.21.15.26:0/1300436540 >> 172.21.15.26:0/1300436540 conn(0x55ac0c1e0100 msgr2=0x55ac0c28a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.668+0000 7f05c971df40 1 -- 172.21.15.26:0/1300436540 shutdown_connections 2024-04-26T21:10:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:03.668+0000 7f05c971df40 1 -- 172.21.15.26:0/1300436540 wait complete. 2024-04-26T21:10:03.674 INFO:teuthology.orchestra.run.smithi026.stdout:Y��alphaparent_b 2024-04-26T21:10:03.675 DEBUG:teuthology.orchestra.run.smithi026:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-26T21:10:03.780 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-26T21:10:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628283, 2024-04-26T21:10:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: "ancestors": [ 2024-04-26T21:10:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:10:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: "dirino": 1099511628284, 2024-04-26T21:10:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: "dname": "alpha", 2024-04-26T21:10:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 3 2024-04-26T21:10:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:10:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:10:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: "dirino": 1, 2024-04-26T21:10:03.780 INFO:teuthology.orchestra.run.smithi026.stdout: "dname": "parent_b", 2024-04-26T21:10:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 25 2024-04-26T21:10:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:10:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-26T21:10:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: "pool": 5, 2024-04-26T21:10:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: "old_pools": [] 2024-04-26T21:10:03.781 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-26T21:10:03.781 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:03.786 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create data_new --pg_num_min 64 2024-04-26T21:10:04.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.003+0000 7fbabfe00640 1 Processor -- start 2024-04-26T21:10:04.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.003+0000 7fbabfe00640 1 -- start start 2024-04-26T21:10:04.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.005+0000 7fbabfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbac0161390 0x7fbac0163ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:04.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.005+0000 7fbabfe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbac0072820 0x7fbac00ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:04.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.005+0000 7fbabfe00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbac006fbe0 con 0x7fbac006e2f0 2024-04-26T21:10:04.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.005+0000 7fbabfe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbac0161770 con 0x7fbac0161390 2024-04-26T21:10:04.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.005+0000 7fbabfe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbac006f750 con 0x7fbac0072820 2024-04-26T21:10:04.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.005+0000 7fbabe000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbac006e2f0 0x7fbac0072110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55472/0 (socket says 172.21.15.26:55472) 2024-04-26T21:10:04.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.005+0000 7fbabe000640 1 -- 172.21.15.26:0/780144421 learned_addr learned my addr 172.21.15.26:0/780144421 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:04.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.005+0000 7fbabea00640 1 --2- 172.21.15.26:0/780144421 >> v2:172.21.15.86:3300/0 conn(0x7fbac0161390 0x7fbac0163ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:04.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.006+0000 7fbabf400640 1 --2- 172.21.15.26:0/780144421 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbac0072820 0x7fbac00ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:04.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.006+0000 7fbabd600640 1 -- 172.21.15.26:0/780144421 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 597906282 0 0) 0x7fbac006fbe0 con 0x7fbac006e2f0 2024-04-26T21:10:04.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.006+0000 7fbabd600640 1 -- 172.21.15.26:0/780144421 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbaa4003520 con 0x7fbac006e2f0 2024-04-26T21:10:04.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.006+0000 7fbabd600640 1 -- 172.21.15.26:0/780144421 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3784683239 0 0) 0x7fbaa4003520 con 0x7fbac006e2f0 2024-04-26T21:10:04.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.006+0000 7fbabea00640 1 -- 172.21.15.26:0/780144421 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbac0072820 msgr2=0x7fbac00ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:04.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.006+0000 7fbabea00640 1 --2- 172.21.15.26:0/780144421 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbac0072820 0x7fbac00ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:04.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.006+0000 7fbabea00640 1 -- 172.21.15.26:0/780144421 >> v1:172.21.15.26:6789/0 conn(0x7fbac006e2f0 legacy=0x7fbac0072110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:04.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.006+0000 7fbabea00640 1 -- 172.21.15.26:0/780144421 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac006eb40 con 0x7fbac0161390 2024-04-26T21:10:04.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.008+0000 7fbabea00640 1 --2- 172.21.15.26:0/780144421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac0161390 0x7fbac0163ce0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbab00097a0 tx=0x7fbab002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a540b19bdf9f096 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:04.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.009+0000 7fbabd600640 1 -- 172.21.15.26:0/780144421 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0037070 con 0x7fbac0161390 2024-04-26T21:10:04.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.009+0000 7fbabd600640 1 -- 172.21.15.26:0/780144421 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab002aba0 con 0x7fbac0161390 2024-04-26T21:10:04.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.009+0000 7fbabd600640 1 -- 172.21.15.26:0/780144421 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac001670 con 0x7fbac0161390 2024-04-26T21:10:04.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.009+0000 7fbabfe00640 1 -- 172.21.15.26:0/780144421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac0161390 msgr2=0x7fbac0163ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:04.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.009+0000 7fbabfe00640 1 --2- 172.21.15.26:0/780144421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac0161390 0x7fbac0163ce0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbab00097a0 tx=0x7fbab002a110 comp rx=0 tx=0).stop 2024-04-26T21:10:04.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.009+0000 7fbabfe00640 1 -- 172.21.15.26:0/780144421 shutdown_connections 2024-04-26T21:10:04.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.009+0000 7fbabfe00640 1 --2- 172.21.15.26:0/780144421 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbac0072820 0x7fbac00ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:04.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.009+0000 7fbabfe00640 1 --2- 172.21.15.26:0/780144421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac0161390 0x7fbac0163ce0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:04.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.009+0000 7fbabfe00640 1 -- 172.21.15.26:0/780144421 >> 172.21.15.26:0/780144421 conn(0x7fbac00b2a10 msgr2=0x7fbac00b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:04.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.009+0000 7fbabfe00640 1 -- 172.21.15.26:0/780144421 shutdown_connections 2024-04-26T21:10:04.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.010+0000 7fbabfe00640 1 -- 172.21.15.26:0/780144421 wait complete. 2024-04-26T21:10:04.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.011+0000 7fbabfe00640 1 Processor -- start 2024-04-26T21:10:04.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.011+0000 7fbabfe00640 1 -- start start 2024-04-26T21:10:04.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.012+0000 7fbabfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbac006e2f0 0x7fbac014bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:04.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.012+0000 7fbabfe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbac0072820 0x7fbac00b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:04.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.012+0000 7fbabfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac0161390 0x7fbac01494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:04.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.012+0000 7fbabfe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbac006f3e0 con 0x7fbac006e2f0 2024-04-26T21:10:04.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.012+0000 7fbabfe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbac006f100 con 0x7fbac0161390 2024-04-26T21:10:04.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.012+0000 7fbabfe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fbac006e780 con 0x7fbac0072820 2024-04-26T21:10:04.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.013+0000 7fbabea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbac006e2f0 0x7fbac014bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:04.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.013+0000 7fbabea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbac006e2f0 0x7fbac014bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47534/0 (socket says 172.21.15.26:47534) 2024-04-26T21:10:04.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.013+0000 7fbabea00640 1 -- 172.21.15.26:0/3839517691 learned_addr learned my addr 172.21.15.26:0/3839517691 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:04.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.013+0000 7fbabea00640 1 -- 172.21.15.26:0/3839517691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbac0072820 msgr2=0x7fbac00b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:04.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.013+0000 7fbabf400640 1 --2- 172.21.15.26:0/3839517691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac0161390 0x7fbac01494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:04.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.013+0000 7fbabea00640 1 --2- 172.21.15.26:0/3839517691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbac0072820 0x7fbac00b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:04.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.013+0000 7fbabea00640 1 -- 172.21.15.26:0/3839517691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac0161390 msgr2=0x7fbac01494e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:04.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.013+0000 7fbabea00640 1 --2- 172.21.15.26:0/3839517691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac0161390 0x7fbac01494e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:04.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.013+0000 7fbabea00640 1 -- 172.21.15.26:0/3839517691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac00b5b60 con 0x7fbac006e2f0 2024-04-26T21:10:04.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.014+0000 7fbabea00640 1 --2- 172.21.15.26:0/3839517691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbac006e2f0 0x7fbac014bfb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbab0000c00 tx=0x7fbab00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:04.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.014+0000 7fbab6a00640 1 -- 172.21.15.26:0/3839517691 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0037040 con 0x7fbac006e2f0 2024-04-26T21:10:04.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.014+0000 7fbab6a00640 1 -- 172.21.15.26:0/3839517691 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab0004030 con 0x7fbac006e2f0 2024-04-26T21:10:04.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.014+0000 7fbab6a00640 1 -- 172.21.15.26:0/3839517691 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0004420 con 0x7fbac006e2f0 2024-04-26T21:10:04.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.014+0000 7fbabfe00640 1 -- 172.21.15.26:0/3839517691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac00b5e40 con 0x7fbac006e2f0 2024-04-26T21:10:04.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.014+0000 7fbabfe00640 1 -- 172.21.15.26:0/3839517691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac00b6400 con 0x7fbac006e2f0 2024-04-26T21:10:04.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.015+0000 7fbabfe00640 1 -- 172.21.15.26:0/3839517691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac006a700 con 0x7fbac006e2f0 2024-04-26T21:10:04.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.016+0000 7fbab6a00640 1 -- 172.21.15.26:0/3839517691 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbab0003610 con 0x7fbac006e2f0 2024-04-26T21:10:04.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.018+0000 7fbab6a00640 1 --2- 172.21.15.26:0/3839517691 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fbaa006c8f0 0x7fbaa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:04.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.018+0000 7fbab6a00640 1 -- 172.21.15.26:0/3839517691 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fbab00ac690 con 0x7fbac006e2f0 2024-04-26T21:10:04.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.018+0000 7fbabe000640 1 --2- 172.21.15.26:0/3839517691 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fbaa006c8f0 0x7fbaa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:04.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.019+0000 7fbabe000640 1 --2- 172.21.15.26:0/3839517691 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fbaa006c8f0 0x7fbaa006edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbaac005880 tx=0x7fbaac001080 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:04.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.023+0000 7fbab6a00640 1 -- 172.21.15.26:0/3839517691 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbab007aab0 con 0x7fbac006e2f0 2024-04-26T21:10:04.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.230+0000 7fbabfe00640 1 -- 172.21.15.26:0/3839517691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64} v 0) -- 0x7fbac00b3670 con 0x7fbac006e2f0 2024-04-26T21:10:04.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.797+0000 7fbab6a00640 1 -- 172.21.15.26:0/3839517691 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64}]=0 pool 'data_new' created v35) ==== 124+0+0 (secure 0 0 0) 0x7fbab007a3b0 con 0x7fbac006e2f0 2024-04-26T21:10:04.799 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'data_new' created 2024-04-26T21:10:04.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.801+0000 7fbabfe00640 1 -- 172.21.15.26:0/3839517691 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fbaa006c8f0 msgr2=0x7fbaa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:04.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.801+0000 7fbabfe00640 1 --2- 172.21.15.26:0/3839517691 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fbaa006c8f0 0x7fbaa006edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbaac005880 tx=0x7fbaac001080 comp rx=0 tx=0).stop 2024-04-26T21:10:04.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.801+0000 7fbabfe00640 1 -- 172.21.15.26:0/3839517691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbac006e2f0 msgr2=0x7fbac014bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:04.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.801+0000 7fbabfe00640 1 --2- 172.21.15.26:0/3839517691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbac006e2f0 0x7fbac014bfb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbab0000c00 tx=0x7fbab00046c0 comp rx=0 tx=0).stop 2024-04-26T21:10:04.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.801+0000 7fbabfe00640 1 -- 172.21.15.26:0/3839517691 shutdown_connections 2024-04-26T21:10:04.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.801+0000 7fbabfe00640 1 --2- 172.21.15.26:0/3839517691 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fbaa006c8f0 0x7fbaa006edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:04.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.801+0000 7fbabfe00640 1 --2- 172.21.15.26:0/3839517691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac0161390 0x7fbac01494e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:04.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.801+0000 7fbabfe00640 1 --2- 172.21.15.26:0/3839517691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbac0072820 0x7fbac00b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:04.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.801+0000 7fbabfe00640 1 --2- 172.21.15.26:0/3839517691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbac006e2f0 0x7fbac014bfb0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:04.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.801+0000 7fbabfe00640 1 -- 172.21.15.26:0/3839517691 >> 172.21.15.26:0/3839517691 conn(0x7fbac00b2a10 msgr2=0x7fbac0161900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:04.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.802+0000 7fbabfe00640 1 -- 172.21.15.26:0/3839517691 shutdown_connections 2024-04-26T21:10:04.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:04.802+0000 7fbabfe00640 1 -- 172.21.15.26:0/3839517691 wait complete. 2024-04-26T21:10:04.833 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs data_new 2024-04-26T21:10:05.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.021+0000 7f80d0e00640 1 Processor -- start 2024-04-26T21:10:05.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.022+0000 7f80d0e00640 1 -- start start 2024-04-26T21:10:05.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.023+0000 7f80d0e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80cc0cef80 0x7f80cc06e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:05.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.023+0000 7f80d0e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80cc163c90 con 0x7f80cc1598f0 2024-04-26T21:10:05.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.023+0000 7f80d0e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80cc0ae980 con 0x7f80cc159510 2024-04-26T21:10:05.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.023+0000 7f80d0e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f80cc06e750 con 0x7f80cc0cef80 2024-04-26T21:10:05.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.023+0000 7f80cbe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f80cc1598f0 0x7f80cc15fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55484/0 (socket says 172.21.15.26:55484) 2024-04-26T21:10:05.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.023+0000 7f80cbe00640 1 -- 172.21.15.26:0/2466658704 learned_addr learned my addr 172.21.15.26:0/2466658704 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:05.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.024+0000 7f80ca000640 1 -- 172.21.15.26:0/2466658704 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 965964149 0 0) 0x7f80cc163c90 con 0x7f80cc1598f0 2024-04-26T21:10:05.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.024+0000 7f80ca000640 1 -- 172.21.15.26:0/2466658704 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80b0003520 con 0x7f80cc1598f0 2024-04-26T21:10:05.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.024+0000 7f80caa00640 1 --2- 172.21.15.26:0/2466658704 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80cc0cef80 0x7f80cc06e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:05.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.025+0000 7f80ca000640 1 -- 172.21.15.26:0/2466658704 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 97893541 0 0) 0x7f80b0003520 con 0x7f80cc1598f0 2024-04-26T21:10:05.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.025+0000 7f80ca000640 1 -- 172.21.15.26:0/2466658704 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80cc0cef80 msgr2=0x7f80cc06e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:05.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.025+0000 7f80ca000640 1 --2- 172.21.15.26:0/2466658704 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80cc0cef80 0x7f80cc06e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:05.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.025+0000 7f80ca000640 1 -- 172.21.15.26:0/2466658704 >> v1:172.21.15.86:6789/0 conn(0x7f80cc159510 legacy=0x7f80cc0ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:05.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.025+0000 7f80ca000640 1 -- 172.21.15.26:0/2466658704 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80cc0aec20 con 0x7f80cc1598f0 2024-04-26T21:10:05.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.025+0000 7f80ca000640 1 -- 172.21.15.26:0/2466658704 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f80c40019f0 con 0x7f80cc1598f0 2024-04-26T21:10:05.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.025+0000 7f80ca000640 1 -- 172.21.15.26:0/2466658704 >> v1:172.21.15.26:6789/0 conn(0x7f80cc1598f0 legacy=0x7f80cc15fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:05.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.025+0000 7f80ca000640 1 --2- 172.21.15.26:0/2466658704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00039e0 0x7f80b0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:05.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.025+0000 7f80ca000640 1 --2- 172.21.15.26:0/2466658704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80b0004800 0x7f80b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:05.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.025+0000 7f80ca000640 1 --2- 172.21.15.26:0/2466658704 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80cc0cef80 0x7f80b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:05.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.025+0000 7f80ca000640 1 -- 172.21.15.26:0/2466658704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc06e750 con 0x7f80b00039e0 2024-04-26T21:10:05.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.025+0000 7f80ca000640 1 -- 172.21.15.26:0/2466658704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80b0005520 con 0x7f80b0004800 2024-04-26T21:10:05.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.025+0000 7f80ca000640 1 -- 172.21.15.26:0/2466658704 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f80b000bf00 con 0x7f80cc0cef80 2024-04-26T21:10:05.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.025+0000 7f80cb400640 1 --2- 172.21.15.26:0/2466658704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00039e0 0x7f80b0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:05.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.025+0000 7f80caa00640 1 --2- 172.21.15.26:0/2466658704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80b0004800 0x7f80b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:05.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.025+0000 7f80cbe00640 1 --2- 172.21.15.26:0/2466658704 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80cc0cef80 0x7f80b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:05.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.026+0000 7f80cb400640 1 -- 172.21.15.26:0/2466658704 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80cc0cef80 msgr2=0x7f80b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:05.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.026+0000 7f80cb400640 1 --2- 172.21.15.26:0/2466658704 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80cc0cef80 0x7f80b000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:05.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.026+0000 7f80cb400640 1 -- 172.21.15.26:0/2466658704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80b0004800 msgr2=0x7f80b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:05.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.026+0000 7f80cb400640 1 --2- 172.21.15.26:0/2466658704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80b0004800 0x7f80b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:05.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.026+0000 7f80cb400640 1 -- 172.21.15.26:0/2466658704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b0003520 con 0x7f80b00039e0 2024-04-26T21:10:05.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.026+0000 7f80caa00640 1 --2- 172.21.15.26:0/2466658704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80b0004800 0x7f80b0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:10:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.028+0000 7f80cb400640 1 --2- 172.21.15.26:0/2466658704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00039e0 0x7f80b0003dc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f80bc001830 tx=0x7f80b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2254ce09521de51 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.028+0000 7f80ca000640 1 -- 172.21.15.26:0/2466658704 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc001430 con 0x7f80b00039e0 2024-04-26T21:10:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.028+0000 7f80ca000640 1 -- 172.21.15.26:0/2466658704 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80bc0052f0 con 0x7f80b00039e0 2024-04-26T21:10:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.028+0000 7f80ca000640 1 -- 172.21.15.26:0/2466658704 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc02ac10 con 0x7f80b00039e0 2024-04-26T21:10:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.028+0000 7f80d0e00640 1 -- 172.21.15.26:0/2466658704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00039e0 msgr2=0x7f80b0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.028+0000 7f80d0e00640 1 --2- 172.21.15.26:0/2466658704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00039e0 0x7f80b0003dc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f80bc001830 tx=0x7f80b0005830 comp rx=0 tx=0).stop 2024-04-26T21:10:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.028+0000 7f80cbe00640 1 -- 172.21.15.26:0/2466658704 reap_dead start 2024-04-26T21:10:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.029+0000 7f80d0e00640 1 -- 172.21.15.26:0/2466658704 shutdown_connections 2024-04-26T21:10:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.029+0000 7f80d0e00640 1 -- 172.21.15.26:0/2466658704 >> 172.21.15.26:0/2466658704 conn(0x7f80cc0b2a10 msgr2=0x7f80cc0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.029+0000 7f80d0e00640 1 -- 172.21.15.26:0/2466658704 shutdown_connections 2024-04-26T21:10:05.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.029+0000 7f80d0e00640 1 -- 172.21.15.26:0/2466658704 wait complete. 2024-04-26T21:10:05.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.031+0000 7f80d0e00640 1 Processor -- start 2024-04-26T21:10:05.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.031+0000 7f80d0e00640 1 -- start start 2024-04-26T21:10:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.032+0000 7f80d0e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00039e0 0x7f80cc0b6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.032+0000 7f80d0e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80cc14af10 0x7f80cc1e9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.032+0000 7f80d0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80cc1e9850 0x7f80cc1f5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.032+0000 7f80d0e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc142420 con 0x7f80b00039e0 2024-04-26T21:10:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.032+0000 7f80d0e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc0af1e0 con 0x7f80cc1e9850 2024-04-26T21:10:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.032+0000 7f80d0e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f80cc06e210 con 0x7f80cc14af10 2024-04-26T21:10:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.032+0000 7f80cb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00039e0 0x7f80cc0b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.032+0000 7f80cb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00039e0 0x7f80cc0b6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47546/0 (socket says 172.21.15.26:47546) 2024-04-26T21:10:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.032+0000 7f80cb400640 1 -- 172.21.15.26:0/810144839 learned_addr learned my addr 172.21.15.26:0/810144839 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.032+0000 7f80caa00640 1 --2- 172.21.15.26:0/810144839 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80cc14af10 0x7f80cc1e9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.032+0000 7f80cbe00640 1 --2- 172.21.15.26:0/810144839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80cc1e9850 0x7f80cc1f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.032+0000 7f80cb400640 1 -- 172.21.15.26:0/810144839 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80cc14af10 msgr2=0x7f80cc1e9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.032+0000 7f80cb400640 1 --2- 172.21.15.26:0/810144839 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80cc14af10 0x7f80cc1e9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.033+0000 7f80cb400640 1 -- 172.21.15.26:0/810144839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80cc1e9850 msgr2=0x7f80cc1f5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.033+0000 7f80cb400640 1 --2- 172.21.15.26:0/810144839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80cc1e9850 0x7f80cc1f5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.033+0000 7f80cb400640 1 -- 172.21.15.26:0/810144839 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80cc0b5570 con 0x7f80b00039e0 2024-04-26T21:10:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.033+0000 7f80cbe00640 1 --2- 172.21.15.26:0/810144839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80cc1e9850 0x7f80cc1f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.033+0000 7f80cb400640 1 --2- 172.21.15.26:0/810144839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00039e0 0x7f80cc0b6610 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f80bc001950 tx=0x7f80bc001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.033+0000 7f80c3400640 1 -- 172.21.15.26:0/810144839 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc03f020 con 0x7f80b00039e0 2024-04-26T21:10:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.033+0000 7f80c3400640 1 -- 172.21.15.26:0/810144839 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80bc0054a0 con 0x7f80b00039e0 2024-04-26T21:10:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.033+0000 7f80c3400640 1 -- 172.21.15.26:0/810144839 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc037040 con 0x7f80b00039e0 2024-04-26T21:10:05.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.033+0000 7f80d0e00640 1 -- 172.21.15.26:0/810144839 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80cc0b5850 con 0x7f80b00039e0 2024-04-26T21:10:05.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.033+0000 7f80d0e00640 1 -- 172.21.15.26:0/810144839 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80cc0b5e10 con 0x7f80b00039e0 2024-04-26T21:10:05.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.034+0000 7f80d0e00640 1 -- 172.21.15.26:0/810144839 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80cc06a700 con 0x7f80b00039e0 2024-04-26T21:10:05.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.036+0000 7f80c3400640 1 -- 172.21.15.26:0/810144839 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80bc042020 con 0x7f80b00039e0 2024-04-26T21:10:05.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.037+0000 7f80c3400640 1 --2- 172.21.15.26:0/810144839 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f80ac06c970 0x7f80ac06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:05.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.037+0000 7f80c3400640 1 -- 172.21.15.26:0/810144839 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6072+0+0 (secure 0 0 0) 0x7f80bc0abe70 con 0x7f80b00039e0 2024-04-26T21:10:05.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.037+0000 7f80caa00640 1 --2- 172.21.15.26:0/810144839 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f80ac06c970 0x7f80ac06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:05.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.038+0000 7f80caa00640 1 --2- 172.21.15.26:0/810144839 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f80ac06c970 0x7f80ac06ee30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f80cc06fbf0 tx=0x7f80b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.041+0000 7f80c3400640 1 -- 172.21.15.26:0/810144839 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f80bc049030 con 0x7f80b00039e0 2024-04-26T21:10:05.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.293+0000 7f80d0e00640 1 -- 172.21.15.26:0/810144839 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"} v 0) -- 0x7f80cc1e9c30 con 0x7f80b00039e0 2024-04-26T21:10:05.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.823+0000 7f80c3400640 1 -- 172.21.15.26:0/810144839 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"}]=0 added data pool 6 to fsmap v18) ==== 131+0+0 (secure 0 0 0) 0x7f80bc07a110 con 0x7f80b00039e0 2024-04-26T21:10:05.832 INFO:teuthology.orchestra.run.smithi026.stderr:added data pool 6 to fsmap 2024-04-26T21:10:05.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.831+0000 7f80d0e00640 1 -- 172.21.15.26:0/810144839 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f80ac06c970 msgr2=0x7f80ac06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:05.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.831+0000 7f80d0e00640 1 --2- 172.21.15.26:0/810144839 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f80ac06c970 0x7f80ac06ee30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f80cc06fbf0 tx=0x7f80b8008040 comp rx=0 tx=0).stop 2024-04-26T21:10:05.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.832+0000 7f80d0e00640 1 -- 172.21.15.26:0/810144839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00039e0 msgr2=0x7f80cc0b6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:05.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.832+0000 7f80d0e00640 1 --2- 172.21.15.26:0/810144839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00039e0 0x7f80cc0b6610 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f80bc001950 tx=0x7f80bc001230 comp rx=0 tx=0).stop 2024-04-26T21:10:05.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.833+0000 7f80d0e00640 1 -- 172.21.15.26:0/810144839 shutdown_connections 2024-04-26T21:10:05.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.833+0000 7f80d0e00640 1 --2- 172.21.15.26:0/810144839 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f80ac06c970 0x7f80ac06ee30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:05.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.833+0000 7f80d0e00640 1 --2- 172.21.15.26:0/810144839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80cc1e9850 0x7f80cc1f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:05.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.833+0000 7f80d0e00640 1 --2- 172.21.15.26:0/810144839 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80cc14af10 0x7f80cc1e9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:05.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.833+0000 7f80d0e00640 1 --2- 172.21.15.26:0/810144839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00039e0 0x7f80cc0b6610 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:05.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.833+0000 7f80d0e00640 1 -- 172.21.15.26:0/810144839 >> 172.21.15.26:0/810144839 conn(0x7f80cc0b2a10 msgr2=0x7f80cc140200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:05.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.834+0000 7f80d0e00640 1 -- 172.21.15.26:0/810144839 shutdown_connections 2024-04-26T21:10:05.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:05.835+0000 7f80d0e00640 1 -- 172.21.15.26:0/810144839 wait complete. 2024-04-26T21:10:05.859 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:10:06.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.037+0000 7f8a8b200640 1 Processor -- start 2024-04-26T21:10:06.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.037+0000 7f8a8b200640 1 -- start start 2024-04-26T21:10:06.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.039+0000 7f8a8b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8a84159b50 0x7f8a8415f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:06.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.039+0000 7f8a8b200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8a8415fdd0 0x7f8a840d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:06.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.039+0000 7f8a8b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a84160440 con 0x7f8a84159770 2024-04-26T21:10:06.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.039+0000 7f8a8b200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f8a84160200 con 0x7f8a8415fdd0 2024-04-26T21:10:06.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.039+0000 7f8a8b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8a84160720 con 0x7f8a84159b50 2024-04-26T21:10:06.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.040+0000 7f8a8a800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8a8415fdd0 0x7f8a840d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:06.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.040+0000 7f8a8a800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8a8415fdd0 0x7f8a840d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47554/0 (socket says 172.21.15.26:47554) 2024-04-26T21:10:06.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.040+0000 7f8a8a800640 1 -- 172.21.15.26:0/3105236520 learned_addr learned my addr 172.21.15.26:0/3105236520 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:06.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.040+0000 7f8a89400640 1 --2- 172.21.15.26:0/3105236520 >> v2:172.21.15.86:3300/0 conn(0x7f8a84159b50 0x7f8a8415f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:06.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.040+0000 7f8a8a800640 1 -- 172.21.15.26:0/3105236520 >> v2:172.21.15.86:3300/0 conn(0x7f8a84159b50 msgr2=0x7f8a8415f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:06.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.040+0000 7f8a8a800640 1 --2- 172.21.15.26:0/3105236520 >> v2:172.21.15.86:3300/0 conn(0x7f8a84159b50 0x7f8a8415f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.040+0000 7f8a8a800640 1 -- 172.21.15.26:0/3105236520 >> v1:172.21.15.86:6789/0 conn(0x7f8a84159770 legacy=0x7f8a840b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:06.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.040+0000 7f8a8a800640 1 -- 172.21.15.26:0/3105236520 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a84072560 con 0x7f8a8415fdd0 2024-04-26T21:10:06.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.041+0000 7f8a8a800640 1 --2- 172.21.15.26:0/3105236520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a8415fdd0 0x7f8a840d1130 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8a84070270 tx=0x7f8a8002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba6b9b76f3b459e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:06.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.042+0000 7f8a88a00640 1 -- 172.21.15.26:0/3105236520 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a80036070 con 0x7f8a8415fdd0 2024-04-26T21:10:06.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.042+0000 7f8a88a00640 1 -- 172.21.15.26:0/3105236520 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a80031600 con 0x7f8a8415fdd0 2024-04-26T21:10:06.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.042+0000 7f8a88a00640 1 -- 172.21.15.26:0/3105236520 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a800319d0 con 0x7f8a8415fdd0 2024-04-26T21:10:06.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.042+0000 7f8a8b200640 1 -- 172.21.15.26:0/3105236520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a8415fdd0 msgr2=0x7f8a840d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:06.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.042+0000 7f8a8b200640 1 --2- 172.21.15.26:0/3105236520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a8415fdd0 0x7f8a840d1130 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8a84070270 tx=0x7f8a8002a6a0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.043+0000 7f8a8b200640 1 -- 172.21.15.26:0/3105236520 shutdown_connections 2024-04-26T21:10:06.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.043+0000 7f8a8b200640 1 --2- 172.21.15.26:0/3105236520 >> v2:172.21.15.86:3300/0 conn(0x7f8a84159b50 0x7f8a8415f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.043+0000 7f8a8b200640 1 --2- 172.21.15.26:0/3105236520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a8415fdd0 0x7f8a840d1130 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.043+0000 7f8a8b200640 1 -- 172.21.15.26:0/3105236520 >> 172.21.15.26:0/3105236520 conn(0x7f8a8406c690 msgr2=0x7f8a8406caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:06.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.043+0000 7f8a8b200640 1 -- 172.21.15.26:0/3105236520 shutdown_connections 2024-04-26T21:10:06.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.043+0000 7f8a8b200640 1 -- 172.21.15.26:0/3105236520 wait complete. 2024-04-26T21:10:06.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.044+0000 7f8a8b200640 1 Processor -- start 2024-04-26T21:10:06.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.045+0000 7f8a8b200640 1 -- start start 2024-04-26T21:10:06.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.046+0000 7f8a8b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a84159770 0x7f8a8414c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:06.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.046+0000 7f8a8b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a84159b50 0x7f8a8415ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:06.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.046+0000 7f8a8b200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8a8415fdd0 0x7f8a8414d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:06.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.046+0000 7f8a8b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8a840d1670 con 0x7f8a84159b50 2024-04-26T21:10:06.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.046+0000 7f8a8b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8406ee10 con 0x7f8a84159770 2024-04-26T21:10:06.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.046+0000 7f8a8b200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8a840d17f0 con 0x7f8a8415fdd0 2024-04-26T21:10:06.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.046+0000 7f8a89400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a84159b50 0x7f8a8415ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:06.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.046+0000 7f8a89400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a84159b50 0x7f8a8415ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47570/0 (socket says 172.21.15.26:47570) 2024-04-26T21:10:06.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.046+0000 7f8a89400640 1 -- 172.21.15.26:0/1401773831 learned_addr learned my addr 172.21.15.26:0/1401773831 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:06.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.046+0000 7f8a89e00640 1 --2- 172.21.15.26:0/1401773831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a84159770 0x7f8a8414c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:06.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.046+0000 7f8a8a800640 1 --2- 172.21.15.26:0/1401773831 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8a8415fdd0 0x7f8a8414d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:06.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.046+0000 7f8a89400640 1 -- 172.21.15.26:0/1401773831 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8a8415fdd0 msgr2=0x7f8a8414d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:06.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.046+0000 7f8a89400640 1 --2- 172.21.15.26:0/1401773831 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8a8415fdd0 0x7f8a8414d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.046+0000 7f8a89400640 1 -- 172.21.15.26:0/1401773831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a84159770 msgr2=0x7f8a8414c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:06.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.046+0000 7f8a89400640 1 --2- 172.21.15.26:0/1401773831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a84159770 0x7f8a8414c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.046+0000 7f8a89400640 1 -- 172.21.15.26:0/1401773831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a8415d430 con 0x7f8a84159b50 2024-04-26T21:10:06.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.046+0000 7f8a89e00640 1 --2- 172.21.15.26:0/1401773831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a84159770 0x7f8a8414c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:06.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.047+0000 7f8a89400640 1 --2- 172.21.15.26:0/1401773831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a84159b50 0x7f8a8415ca60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400d800 tx=0x7f8a7400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:06.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.047+0000 7f8a7e000640 1 -- 172.21.15.26:0/1401773831 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7400b6c0 con 0x7f8a84159b50 2024-04-26T21:10:06.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.047+0000 7f8a7e000640 1 -- 172.21.15.26:0/1401773831 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a74019070 con 0x7f8a84159b50 2024-04-26T21:10:06.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.047+0000 7f8a7e000640 1 -- 172.21.15.26:0/1401773831 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7400baa0 con 0x7f8a84159b50 2024-04-26T21:10:06.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.047+0000 7f8a8b200640 1 -- 172.21.15.26:0/1401773831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a8415d710 con 0x7f8a84159b50 2024-04-26T21:10:06.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.047+0000 7f8a8b200640 1 -- 172.21.15.26:0/1401773831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a8415dcd0 con 0x7f8a84159b50 2024-04-26T21:10:06.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.049+0000 7f8a8b200640 1 -- 172.21.15.26:0/1401773831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a8406a700 con 0x7f8a84159b50 2024-04-26T21:10:06.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.049+0000 7f8a7e000640 1 -- 172.21.15.26:0/1401773831 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a7400bc40 con 0x7f8a84159b50 2024-04-26T21:10:06.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.050+0000 7f8a7e000640 1 --2- 172.21.15.26:0/1401773831 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8a6806c920 0x7f8a6806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:06.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.050+0000 7f8a7e000640 1 -- 172.21.15.26:0/1401773831 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6104+0+0 (secure 0 0 0) 0x7f8a7408c180 con 0x7f8a84159b50 2024-04-26T21:10:06.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.051+0000 7f8a89e00640 1 --2- 172.21.15.26:0/1401773831 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8a6806c920 0x7f8a6806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:06.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.051+0000 7f8a89e00640 1 --2- 172.21.15.26:0/1401773831 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8a6806c920 0x7f8a6806ede0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8a78000f60 tx=0x7f8a78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:06.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.055+0000 7f8a7e000640 1 -- 172.21.15.26:0/1401773831 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a7405a410 con 0x7f8a84159b50 2024-04-26T21:10:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.307+0000 7f8a8b200640 1 -- 172.21.15.26:0/1401773831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a840b39a0 con 0x7f8a84159b50 2024-04-26T21:10:06.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.309+0000 7f8a7e000640 1 -- 172.21.15.26:0/1401773831 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5180 (secure 0 0 0) 0x7f8a74059d10 con 0x7f8a84159b50 2024-04-26T21:10:06.311 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:06.311 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2349160700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2349160700},{"type":"v1","addr":"172.21.15.26:6837","nonce":2349160700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4598,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2824191762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2824191762},{"type":"v1","addr":"172.21.15.86:6834","nonce":2824191762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/263746310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":263746310},{"type":"v1","addr":"172.21.15.86:6835","nonce":263746310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:09:48.622025+0000","modified":"2024-04-26T21:10:05.296141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5058":{"gid":5058,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6836/2528180854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2528180854},{"type":"v1","addr":"172.21.15.26:6836","nonce":2528180854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5058,"qdb_cluster":[5058]},"id":2}]} 2024-04-26T21:10:06.312 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 18 2024-04-26T21:10:06.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.312+0000 7f8a8b200640 1 -- 172.21.15.26:0/1401773831 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8a6806c920 msgr2=0x7f8a6806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:06.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.312+0000 7f8a8b200640 1 --2- 172.21.15.26:0/1401773831 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8a6806c920 0x7f8a6806ede0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8a78000f60 tx=0x7f8a78009040 comp rx=0 tx=0).stop 2024-04-26T21:10:06.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.312+0000 7f8a8b200640 1 -- 172.21.15.26:0/1401773831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a84159b50 msgr2=0x7f8a8415ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:06.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.312+0000 7f8a8b200640 1 --2- 172.21.15.26:0/1401773831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a84159b50 0x7f8a8415ca60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400d800 tx=0x7f8a7400dcc0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.312+0000 7f8a8b200640 1 -- 172.21.15.26:0/1401773831 shutdown_connections 2024-04-26T21:10:06.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.312+0000 7f8a8b200640 1 --2- 172.21.15.26:0/1401773831 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8a6806c920 0x7f8a6806ede0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.312+0000 7f8a8b200640 1 --2- 172.21.15.26:0/1401773831 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8a8415fdd0 0x7f8a8414d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.312+0000 7f8a8b200640 1 --2- 172.21.15.26:0/1401773831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a84159b50 0x7f8a8415ca60 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.312+0000 7f8a8b200640 1 --2- 172.21.15.26:0/1401773831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a84159770 0x7f8a8414c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.312+0000 7f8a8b200640 1 -- 172.21.15.26:0/1401773831 >> 172.21.15.26:0/1401773831 conn(0x7f8a8406c690 msgr2=0x7f8a840b2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:06.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.313+0000 7f8a8b200640 1 -- 172.21.15.26:0/1401773831 shutdown_connections 2024-04-26T21:10:06.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.313+0000 7f8a8b200640 1 -- 172.21.15.26:0/1401773831 wait complete. 2024-04-26T21:10:06.333 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:10:06.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.543+0000 7f8533e00640 1 Processor -- start 2024-04-26T21:10:06.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.544+0000 7f8533e00640 1 -- start start 2024-04-26T21:10:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.545+0000 7f8533e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8534148aa0 0x7f8534163930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.545+0000 7f8533e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f853406ee00 0x7f85340d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.545+0000 7f8533e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f85340d0fa0 0x7f85340d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:06.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.545+0000 7f8533e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f85340c8370 con 0x7f85340d0fa0 2024-04-26T21:10:06.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.545+0000 7f8533e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f85340c7db0 con 0x7f8534148aa0 2024-04-26T21:10:06.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.545+0000 7f8533e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f85340c8090 con 0x7f853406ee00 2024-04-26T21:10:06.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.546+0000 7f8533400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f85340d0fa0 0x7f85340d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:06.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.546+0000 7f8533400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f85340d0fa0 0x7f85340d2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47582/0 (socket says 172.21.15.26:47582) 2024-04-26T21:10:06.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.546+0000 7f8533400640 1 -- 172.21.15.26:0/2846011247 learned_addr learned my addr 172.21.15.26:0/2846011247 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:06.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.546+0000 7f8533400640 1 -- 172.21.15.26:0/2846011247 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f853406ee00 msgr2=0x7f85340d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:06.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.546+0000 7f8532000640 1 --2- 172.21.15.26:0/2846011247 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f853406ee00 0x7f85340d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:06.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.546+0000 7f8532a00640 1 --2- 172.21.15.26:0/2846011247 >> v2:172.21.15.86:3300/0 conn(0x7f8534148aa0 0x7f8534163930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:06.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.546+0000 7f8533400640 1 --2- 172.21.15.26:0/2846011247 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f853406ee00 0x7f85340d0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.546+0000 7f8533400640 1 -- 172.21.15.26:0/2846011247 >> v2:172.21.15.86:3300/0 conn(0x7f8534148aa0 msgr2=0x7f8534163930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:06.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.546+0000 7f8533400640 1 --2- 172.21.15.26:0/2846011247 >> v2:172.21.15.86:3300/0 conn(0x7f8534148aa0 0x7f8534163930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.546+0000 7f8533400640 1 -- 172.21.15.26:0/2846011247 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853406d480 con 0x7f85340d0fa0 2024-04-26T21:10:06.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.547+0000 7f8533400640 1 --2- 172.21.15.26:0/2846011247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85340d0fa0 0x7f85340d2e00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f853406f530 tx=0x7f852c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a6168dab8e49cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:06.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.548+0000 7f8531600640 1 -- 172.21.15.26:0/2846011247 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c036070 con 0x7f85340d0fa0 2024-04-26T21:10:06.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.548+0000 7f8531600640 1 -- 172.21.15.26:0/2846011247 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f852c031600 con 0x7f85340d0fa0 2024-04-26T21:10:06.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.548+0000 7f8531600640 1 -- 172.21.15.26:0/2846011247 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c0319d0 con 0x7f85340d0fa0 2024-04-26T21:10:06.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.548+0000 7f8533e00640 1 -- 172.21.15.26:0/2846011247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85340d0fa0 msgr2=0x7f85340d2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.548+0000 7f8533e00640 1 --2- 172.21.15.26:0/2846011247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85340d0fa0 0x7f85340d2e00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f853406f530 tx=0x7f852c02a610 comp rx=0 tx=0).stop 2024-04-26T21:10:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.548+0000 7f8533e00640 1 -- 172.21.15.26:0/2846011247 shutdown_connections 2024-04-26T21:10:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.549+0000 7f8533e00640 1 --2- 172.21.15.26:0/2846011247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85340d0fa0 0x7f85340d2e00 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.549+0000 7f8533e00640 1 --2- 172.21.15.26:0/2846011247 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f853406ee00 0x7f85340d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.549+0000 7f8533e00640 1 --2- 172.21.15.26:0/2846011247 >> v2:172.21.15.86:3300/0 conn(0x7f8534148aa0 0x7f8534163930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.549+0000 7f8533e00640 1 -- 172.21.15.26:0/2846011247 >> 172.21.15.26:0/2846011247 conn(0x7f8534075960 msgr2=0x7f853406c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.549+0000 7f8533e00640 1 -- 172.21.15.26:0/2846011247 shutdown_connections 2024-04-26T21:10:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.549+0000 7f8533e00640 1 -- 172.21.15.26:0/2846011247 wait complete. 2024-04-26T21:10:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.550+0000 7f8533e00640 1 Processor -- start 2024-04-26T21:10:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.551+0000 7f8533e00640 1 -- start start 2024-04-26T21:10:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.552+0000 7f8533e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f853406ee00 0x7f85341449a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.552+0000 7f8533e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85340d0fa0 0x7f853415da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.552+0000 7f8533e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8534148aa0 0x7f853415df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.552+0000 7f8533e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f853406f280 con 0x7f853406ee00 2024-04-26T21:10:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.552+0000 7f8533e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8534148e80 con 0x7f85340d0fa0 2024-04-26T21:10:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.552+0000 7f8533e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f85340d3340 con 0x7f8534148aa0 2024-04-26T21:10:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.552+0000 7f8532a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f853406ee00 0x7f85341449a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.552+0000 7f8532a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f853406ee00 0x7f85341449a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47586/0 (socket says 172.21.15.26:47586) 2024-04-26T21:10:06.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.552+0000 7f8532a00640 1 -- 172.21.15.26:0/1511978968 learned_addr learned my addr 172.21.15.26:0/1511978968 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:06.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.552+0000 7f8532000640 1 --2- 172.21.15.26:0/1511978968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85340d0fa0 0x7f853415da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:06.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.552+0000 7f8533400640 1 --2- 172.21.15.26:0/1511978968 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8534148aa0 0x7f853415df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:06.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.552+0000 7f8532a00640 1 -- 172.21.15.26:0/1511978968 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8534148aa0 msgr2=0x7f853415df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:06.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.552+0000 7f8532a00640 1 --2- 172.21.15.26:0/1511978968 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8534148aa0 0x7f853415df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.552+0000 7f8532a00640 1 -- 172.21.15.26:0/1511978968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85340d0fa0 msgr2=0x7f853415da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:06.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.552+0000 7f8532a00640 1 --2- 172.21.15.26:0/1511978968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85340d0fa0 0x7f853415da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.552+0000 7f8532a00640 1 -- 172.21.15.26:0/1511978968 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85340b5f50 con 0x7f853406ee00 2024-04-26T21:10:06.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.553+0000 7f8532a00640 1 --2- 172.21.15.26:0/1511978968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f853406ee00 0x7f85341449a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f852400c7f0 tx=0x7f852400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:06.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.553+0000 7f852aa00640 1 -- 172.21.15.26:0/1511978968 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8524015070 con 0x7f853406ee00 2024-04-26T21:10:06.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.553+0000 7f852aa00640 1 -- 172.21.15.26:0/1511978968 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8524004220 con 0x7f853406ee00 2024-04-26T21:10:06.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.553+0000 7f852aa00640 1 -- 172.21.15.26:0/1511978968 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8524007620 con 0x7f853406ee00 2024-04-26T21:10:06.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.553+0000 7f8533e00640 1 -- 172.21.15.26:0/1511978968 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85340b6230 con 0x7f853406ee00 2024-04-26T21:10:06.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.553+0000 7f8533e00640 1 -- 172.21.15.26:0/1511978968 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85340b67f0 con 0x7f853406ee00 2024-04-26T21:10:06.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.554+0000 7f8533e00640 1 -- 172.21.15.26:0/1511978968 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8534162960 con 0x7f853406ee00 2024-04-26T21:10:06.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.555+0000 7f852aa00640 1 -- 172.21.15.26:0/1511978968 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f85240077c0 con 0x7f853406ee00 2024-04-26T21:10:06.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.556+0000 7f852aa00640 1 --2- 172.21.15.26:0/1511978968 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f851006c8f0 0x7f851006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:06.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.556+0000 7f852aa00640 1 -- 172.21.15.26:0/1511978968 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6104+0+0 (secure 0 0 0) 0x7f852408bf30 con 0x7f853406ee00 2024-04-26T21:10:06.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.556+0000 7f8532000640 1 --2- 172.21.15.26:0/1511978968 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f851006c8f0 0x7f851006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:06.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.557+0000 7f8532000640 1 --2- 172.21.15.26:0/1511978968 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f851006c8f0 0x7f851006edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f85340b8b70 tx=0x7f851c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:06.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.561+0000 7f852aa00640 1 -- 172.21.15.26:0/1511978968 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8534162960 con 0x7f853406ee00 2024-04-26T21:10:06.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.756+0000 7f8533e00640 1 -- 172.21.15.26:0/1511978968 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85340693d0 con 0x7f853406ee00 2024-04-26T21:10:06.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.759+0000 7f852aa00640 1 -- 172.21.15.26:0/1511978968 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+18446 (secure 0 0 0) 0x7f852405a130 con 0x7f853406ee00 2024-04-26T21:10:06.761 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:06.761 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":36,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:10:05.295617+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:46.646256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:47.613883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-04-26T21:10:04.249730+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:10:06.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.761+0000 7f8533e00640 1 -- 172.21.15.26:0/1511978968 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f851006c8f0 msgr2=0x7f851006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:06.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.761+0000 7f8533e00640 1 --2- 172.21.15.26:0/1511978968 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f851006c8f0 0x7f851006edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f85340b8b70 tx=0x7f851c008040 comp rx=0 tx=0).stop 2024-04-26T21:10:06.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.761+0000 7f8533e00640 1 -- 172.21.15.26:0/1511978968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f853406ee00 msgr2=0x7f85341449a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:06.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.761+0000 7f8533e00640 1 --2- 172.21.15.26:0/1511978968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f853406ee00 0x7f85341449a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f852400c7f0 tx=0x7f852400ccb0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.761+0000 7f8533e00640 1 -- 172.21.15.26:0/1511978968 shutdown_connections 2024-04-26T21:10:06.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.761+0000 7f8533e00640 1 --2- 172.21.15.26:0/1511978968 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f851006c8f0 0x7f851006edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.761+0000 7f8533e00640 1 --2- 172.21.15.26:0/1511978968 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8534148aa0 0x7f853415df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.761+0000 7f8533e00640 1 --2- 172.21.15.26:0/1511978968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85340d0fa0 0x7f853415da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.761+0000 7f8533e00640 1 --2- 172.21.15.26:0/1511978968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f853406ee00 0x7f85341449a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:06.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.761+0000 7f8533e00640 1 -- 172.21.15.26:0/1511978968 >> 172.21.15.26:0/1511978968 conn(0x7f8534075960 msgr2=0x7f853406c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:06.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.762+0000 7f8533e00640 1 -- 172.21.15.26:0/1511978968 shutdown_connections 2024-04-26T21:10:06.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:06.762+0000 7f8533e00640 1 -- 172.21.15.26:0/1511978968 wait complete. 2024-04-26T21:10:06.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:10:06.782 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new ./parent_b/alpha) 2024-04-26T21:10:06.851 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:10:06.996 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:10:07.003 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:10:07.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.044+0000 7f52a4e00640 1 Processor -- start 2024-04-26T21:10:07.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.044+0000 7f52a4e00640 1 -- start start 2024-04-26T21:10:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.045+0000 7f52a4e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f52a0162ba0 0x7f52a00d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.045+0000 7f52a4e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f52a00d2060 0x7f52a00d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.046+0000 7f52a4e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52a00b61c0 con 0x7f52a00d1c80 2024-04-26T21:10:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.046+0000 7f52a4e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f52a00b5f70 con 0x7f52a00d2060 2024-04-26T21:10:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.046+0000 7f52a4e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f52a00d3830 con 0x7f52a0162ba0 2024-04-26T21:10:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.046+0000 7f529f400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f52a00d1c80 0x7f52a0162490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55488/0 (socket says 172.21.15.26:55488) 2024-04-26T21:10:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.046+0000 7f529f400640 1 -- 172.21.15.26:0/1365805384 learned_addr learned my addr 172.21.15.26:0/1365805384 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.046+0000 7f529ea00640 1 --2- 172.21.15.26:0/1365805384 >> v2:172.21.15.86:3300/0 conn(0x7f52a0162ba0 0x7f52a00d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.046+0000 7f529fe00640 1 --2- 172.21.15.26:0/1365805384 >> v2:172.21.15.26:3300/0 conn(0x7f52a00d2060 0x7f52a00d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.047+0000 7f529fe00640 1 -- 172.21.15.26:0/1365805384 >> v2:172.21.15.86:3300/0 conn(0x7f52a0162ba0 msgr2=0x7f52a00d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.047+0000 7f529fe00640 1 --2- 172.21.15.26:0/1365805384 >> v2:172.21.15.86:3300/0 conn(0x7f52a0162ba0 0x7f52a00d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.047+0000 7f529e000640 1 -- 172.21.15.26:0/1365805384 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2403116860 0 0) 0x7f52a00b61c0 con 0x7f52a00d1c80 2024-04-26T21:10:07.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.047+0000 7f529fe00640 1 -- 172.21.15.26:0/1365805384 >> v1:172.21.15.26:6789/0 conn(0x7f52a00d1c80 legacy=0x7f52a0162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.047+0000 7f529fe00640 1 -- 172.21.15.26:0/1365805384 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a00b64a0 con 0x7f52a00d2060 2024-04-26T21:10:07.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.048+0000 7f529fe00640 1 --2- 172.21.15.26:0/1365805384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a00d2060 0x7f52a00d3cc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f52a00d4ff0 tx=0x7f529802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe7712167ceb0d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.048+0000 7f529e000640 1 -- 172.21.15.26:0/1365805384 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298007740 con 0x7f52a00d2060 2024-04-26T21:10:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.048+0000 7f529e000640 1 -- 172.21.15.26:0/1365805384 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52980078e0 con 0x7f52a00d2060 2024-04-26T21:10:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.048+0000 7f529e000640 1 -- 172.21.15.26:0/1365805384 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298007cb0 con 0x7f52a00d2060 2024-04-26T21:10:07.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.049+0000 7f52a4e00640 1 -- 172.21.15.26:0/1365805384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a00d2060 msgr2=0x7f52a00d3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.049+0000 7f52a4e00640 1 --2- 172.21.15.26:0/1365805384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a00d2060 0x7f52a00d3cc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f52a00d4ff0 tx=0x7f529802a5f0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.049+0000 7f52a4e00640 1 -- 172.21.15.26:0/1365805384 shutdown_connections 2024-04-26T21:10:07.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.049+0000 7f52a4e00640 1 --2- 172.21.15.26:0/1365805384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a00d2060 0x7f52a00d3cc0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.049+0000 7f52a4e00640 1 --2- 172.21.15.26:0/1365805384 >> v2:172.21.15.86:3300/0 conn(0x7f52a0162ba0 0x7f52a00d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.049+0000 7f52a4e00640 1 -- 172.21.15.26:0/1365805384 >> 172.21.15.26:0/1365805384 conn(0x7f52a00d5020 msgr2=0x7f52a00d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:07.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.049+0000 7f52a4e00640 1 -- 172.21.15.26:0/1365805384 shutdown_connections 2024-04-26T21:10:07.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.049+0000 7f52a4e00640 1 -- 172.21.15.26:0/1365805384 wait complete. 2024-04-26T21:10:07.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.052+0000 7f52a4e00640 1 Processor -- start 2024-04-26T21:10:07.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.052+0000 7f52a4e00640 1 -- start start 2024-04-26T21:10:07.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.054+0000 7f52a4e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f52a00d1c80 0x7f52a0153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.054+0000 7f52a4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52a00d2060 0x7f52a00b7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.054+0000 7f52a4e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a0162ba0 0x7f52a00cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.054+0000 7f52a4e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f52a00d5cc0 con 0x7f52a0162ba0 2024-04-26T21:10:07.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.054+0000 7f52a4e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52a00d5b40 con 0x7f52a00d2060 2024-04-26T21:10:07.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.054+0000 7f52a4e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f52a00c8b70 con 0x7f52a00d1c80 2024-04-26T21:10:07.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.054+0000 7f529fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a0162ba0 0x7f52a00cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.054+0000 7f529fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a0162ba0 0x7f52a00cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47608/0 (socket says 172.21.15.26:47608) 2024-04-26T21:10:07.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.054+0000 7f529fe00640 1 -- 172.21.15.26:0/4174265454 learned_addr learned my addr 172.21.15.26:0/4174265454 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:07.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.054+0000 7f529f400640 1 --2- 172.21.15.26:0/4174265454 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f52a00d1c80 0x7f52a0153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.054+0000 7f529ea00640 1 --2- 172.21.15.26:0/4174265454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52a00d2060 0x7f52a00b7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.055+0000 7f529fe00640 1 -- 172.21.15.26:0/4174265454 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f52a00d1c80 msgr2=0x7f52a0153410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.055+0000 7f529fe00640 1 --2- 172.21.15.26:0/4174265454 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f52a00d1c80 0x7f52a0153410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.055+0000 7f529fe00640 1 -- 172.21.15.26:0/4174265454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52a00d2060 msgr2=0x7f52a00b7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.055+0000 7f529fe00640 1 --2- 172.21.15.26:0/4174265454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52a00d2060 0x7f52a00b7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.055+0000 7f529fe00640 1 -- 172.21.15.26:0/4174265454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a01f5c00 con 0x7f52a0162ba0 2024-04-26T21:10:07.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.055+0000 7f529fe00640 1 --2- 172.21.15.26:0/4174265454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a0162ba0 0x7f52a00cd9a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f529802abe0 tx=0x7f5298004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:07.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.055+0000 7f5297400640 1 -- 172.21.15.26:0/4174265454 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298040070 con 0x7f52a0162ba0 2024-04-26T21:10:07.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.055+0000 7f5297400640 1 -- 172.21.15.26:0/4174265454 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5298031bc0 con 0x7f52a0162ba0 2024-04-26T21:10:07.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.055+0000 7f5297400640 1 -- 172.21.15.26:0/4174265454 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298009450 con 0x7f52a0162ba0 2024-04-26T21:10:07.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.055+0000 7f52a4e00640 1 -- 172.21.15.26:0/4174265454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52a00cc290 con 0x7f52a0162ba0 2024-04-26T21:10:07.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.056+0000 7f52a4e00640 1 -- 172.21.15.26:0/4174265454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52a00cc850 con 0x7f52a0162ba0 2024-04-26T21:10:07.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.057+0000 7f52a4e00640 1 -- 172.21.15.26:0/4174265454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52a006a700 con 0x7f52a0162ba0 2024-04-26T21:10:07.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.058+0000 7f5297400640 1 -- 172.21.15.26:0/4174265454 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5298039040 con 0x7f52a0162ba0 2024-04-26T21:10:07.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.059+0000 7f5297400640 1 --2- 172.21.15.26:0/4174265454 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f528006c8f0 0x7f528006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.059+0000 7f5297400640 1 -- 172.21.15.26:0/4174265454 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f52980ad530 con 0x7f52a0162ba0 2024-04-26T21:10:07.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.060+0000 7f529f400640 1 --2- 172.21.15.26:0/4174265454 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f528006c8f0 0x7f528006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.060+0000 7f529f400640 1 --2- 172.21.15.26:0/4174265454 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f528006c8f0 0x7f528006edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5290000f60 tx=0x7f5290007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:07.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.063+0000 7f5297400640 1 -- 172.21.15.26:0/4174265454 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f529807b200 con 0x7f52a0162ba0 2024-04-26T21:10:07.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.310+0000 7f52a4e00640 1 -- 172.21.15.26:0/4174265454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52a00d26a0 con 0x7f52a0162ba0 2024-04-26T21:10:07.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.311+0000 7f5297400640 1 -- 172.21.15.26:0/4174265454 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5180 (secure 0 0 0) 0x7f529807f500 con 0x7f52a0162ba0 2024-04-26T21:10:07.313 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:07.313 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2349160700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2349160700},{"type":"v1","addr":"172.21.15.26:6837","nonce":2349160700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4598,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2824191762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2824191762},{"type":"v1","addr":"172.21.15.86:6834","nonce":2824191762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/263746310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":263746310},{"type":"v1","addr":"172.21.15.86:6835","nonce":263746310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:09:48.622025+0000","modified":"2024-04-26T21:10:05.296141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5058":{"gid":5058,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6836/2528180854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2528180854},{"type":"v1","addr":"172.21.15.26:6836","nonce":2528180854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5058,"qdb_cluster":[5058]},"id":2}]} 2024-04-26T21:10:07.313 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 18 2024-04-26T21:10:07.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.314+0000 7f52a4e00640 1 -- 172.21.15.26:0/4174265454 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f528006c8f0 msgr2=0x7f528006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.314+0000 7f52a4e00640 1 --2- 172.21.15.26:0/4174265454 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f528006c8f0 0x7f528006edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5290000f60 tx=0x7f5290007d50 comp rx=0 tx=0).stop 2024-04-26T21:10:07.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.314+0000 7f52a4e00640 1 -- 172.21.15.26:0/4174265454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a0162ba0 msgr2=0x7f52a00cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.314+0000 7f52a4e00640 1 --2- 172.21.15.26:0/4174265454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a0162ba0 0x7f52a00cd9a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f529802abe0 tx=0x7f5298004a40 comp rx=0 tx=0).stop 2024-04-26T21:10:07.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.314+0000 7f52a4e00640 1 -- 172.21.15.26:0/4174265454 shutdown_connections 2024-04-26T21:10:07.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.314+0000 7f52a4e00640 1 --2- 172.21.15.26:0/4174265454 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f528006c8f0 0x7f528006edb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.314+0000 7f52a4e00640 1 --2- 172.21.15.26:0/4174265454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52a0162ba0 0x7f52a00cd9a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.314+0000 7f52a4e00640 1 --2- 172.21.15.26:0/4174265454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52a00d2060 0x7f52a00b7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.314+0000 7f52a4e00640 1 --2- 172.21.15.26:0/4174265454 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f52a00d1c80 0x7f52a0153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.314+0000 7f52a4e00640 1 -- 172.21.15.26:0/4174265454 >> 172.21.15.26:0/4174265454 conn(0x7f52a00d5020 msgr2=0x7f52a00caf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:07.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.314+0000 7f52a4e00640 1 -- 172.21.15.26:0/4174265454 shutdown_connections 2024-04-26T21:10:07.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.314+0000 7f52a4e00640 1 -- 172.21.15.26:0/4174265454 wait complete. 2024-04-26T21:10:07.336 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-26T21:10:07.586 INFO:teuthology.orchestra.run.smithi026.stdout:{"message":"","return_code":0} 2024-04-26T21:10:07.587 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-26T21:10:07.587 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001fb.00000000 parent 2024-04-26T21:10:07.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.693+0000 7f86c4571f40 1 Processor -- start 2024-04-26T21:10:07.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.693+0000 7f86c4571f40 1 -- start start 2024-04-26T21:10:07.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.694+0000 7f86c4571f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86a80022d0 0x561669fa2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.694+0000 7f86c4571f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56166a07c6a0 0x561669fe9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.694+0000 7f86c4571f40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56166a073980 0x561669fefe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.694+0000 7f86c4571f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561669f49d20 con 0x7f86a80022d0 2024-04-26T21:10:07.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.694+0000 7f86c4571f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561669f4dd60 con 0x56166a07c6a0 2024-04-26T21:10:07.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.694+0000 7f86c4571f40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x56166a07ab80 con 0x56166a073980 2024-04-26T21:10:07.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.694+0000 7f86c1800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86a80022d0 0x561669fa2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.694+0000 7f86c1800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86a80022d0 0x561669fa2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47640/0 (socket says 172.21.15.26:47640) 2024-04-26T21:10:07.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.694+0000 7f86c1800640 1 -- 172.21.15.26:0/1218279290 learned_addr learned my addr 172.21.15.26:0/1218279290 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:07.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.695+0000 7f86c2200640 1 --2- 172.21.15.26:0/1218279290 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56166a073980 0x561669fefe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.695+0000 7f86b8e00640 1 --2- 172.21.15.26:0/1218279290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56166a07c6a0 0x561669fe9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.695+0000 7f86c1800640 1 -- 172.21.15.26:0/1218279290 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56166a073980 msgr2=0x561669fefe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:07.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.695+0000 7f86c1800640 1 --2- 172.21.15.26:0/1218279290 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56166a073980 0x561669fefe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.695+0000 7f86c1800640 1 -- 172.21.15.26:0/1218279290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56166a07c6a0 msgr2=0x561669fe9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:07.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.695+0000 7f86c1800640 1 --2- 172.21.15.26:0/1218279290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56166a07c6a0 0x561669fe9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.695+0000 7f86c1800640 1 -- 172.21.15.26:0/1218279290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561669fd1830 con 0x7f86a80022d0 2024-04-26T21:10:07.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.695+0000 7f86b8e00640 1 --2- 172.21.15.26:0/1218279290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56166a07c6a0 0x561669fe9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:07.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.695+0000 7f86c1800640 1 --2- 172.21.15.26:0/1218279290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86a80022d0 0x561669fa2730 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f86b40259c0 tx=0x7f86b4053000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9625623f88174a69 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:07.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.695+0000 7f86bbe00640 1 -- 172.21.15.26:0/1218279290 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4050a60 con 0x7f86a80022d0 2024-04-26T21:10:07.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.696+0000 7f86bbe00640 1 -- 172.21.15.26:0/1218279290 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86b4050c20 con 0x7f86a80022d0 2024-04-26T21:10:07.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.696+0000 7f86bbe00640 1 -- 172.21.15.26:0/1218279290 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4053480 con 0x7f86a80022d0 2024-04-26T21:10:07.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.696+0000 7f86c4571f40 1 -- 172.21.15.26:0/1218279290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86a80022d0 msgr2=0x561669fa2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.696+0000 7f86c4571f40 1 --2- 172.21.15.26:0/1218279290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86a80022d0 0x561669fa2730 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f86b40259c0 tx=0x7f86b4053000 comp rx=0 tx=0).stop 2024-04-26T21:10:07.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.696+0000 7f86c4571f40 1 -- 172.21.15.26:0/1218279290 shutdown_connections 2024-04-26T21:10:07.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.696+0000 7f86c4571f40 1 --2- 172.21.15.26:0/1218279290 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56166a073980 0x561669fefe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.696+0000 7f86c4571f40 1 --2- 172.21.15.26:0/1218279290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56166a07c6a0 0x561669fe9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.696+0000 7f86c4571f40 1 --2- 172.21.15.26:0/1218279290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86a80022d0 0x561669fa2730 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.696+0000 7f86c4571f40 1 -- 172.21.15.26:0/1218279290 >> 172.21.15.26:0/1218279290 conn(0x561669fd2100 msgr2=0x56166a07bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:07.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.696+0000 7f86c4571f40 1 -- 172.21.15.26:0/1218279290 shutdown_connections 2024-04-26T21:10:07.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.696+0000 7f86c4571f40 1 -- 172.21.15.26:0/1218279290 wait complete. 2024-04-26T21:10:07.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.697+0000 7f86c4571f40 1 Processor -- start 2024-04-26T21:10:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.697+0000 7f86c4571f40 1 -- start start 2024-04-26T21:10:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.698+0000 7f86c4571f40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56166a073980 0x561669fa16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.698+0000 7f86c4571f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166a07c6a0 0x56166a06b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.698+0000 7f86c4571f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86a80022d0 0x56166a06f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.698+0000 7f86c4571f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56166a06cc60 con 0x56166a07c6a0 2024-04-26T21:10:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.698+0000 7f86c4571f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56166a06ca10 con 0x7f86a80022d0 2024-04-26T21:10:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.698+0000 7f86c4571f40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x561669fd5a60 con 0x56166a073980 2024-04-26T21:10:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.698+0000 7f86b8e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166a07c6a0 0x56166a06b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.699+0000 7f86b8e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166a07c6a0 0x56166a06b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47648/0 (socket says 172.21.15.26:47648) 2024-04-26T21:10:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.699+0000 7f86b8e00640 1 -- 172.21.15.26:0/1858672463 learned_addr learned my addr 172.21.15.26:0/1858672463 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.699+0000 7f86c2200640 1 --2- 172.21.15.26:0/1858672463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86a80022d0 0x56166a06f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.699+0000 7f86c1800640 1 --2- 172.21.15.26:0/1858672463 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56166a073980 0x561669fa16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.699+0000 7f86b8e00640 1 -- 172.21.15.26:0/1858672463 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56166a073980 msgr2=0x561669fa16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.699+0000 7f86b8e00640 1 --2- 172.21.15.26:0/1858672463 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56166a073980 0x561669fa16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.699+0000 7f86b8e00640 1 -- 172.21.15.26:0/1858672463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86a80022d0 msgr2=0x56166a06f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.699+0000 7f86b8e00640 1 --2- 172.21.15.26:0/1858672463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86a80022d0 0x56166a06f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.699+0000 7f86b8e00640 1 -- 172.21.15.26:0/1858672463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561669fe7100 con 0x56166a07c6a0 2024-04-26T21:10:07.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.699+0000 7f86c1800640 1 --2- 172.21.15.26:0/1858672463 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56166a073980 0x561669fa16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:07.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.699+0000 7f86b8e00640 1 --2- 172.21.15.26:0/1858672463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166a07c6a0 0x56166a06b920 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f86ac002fc0 tx=0x7f86ac00c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:07.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.700+0000 7f86baa00640 1 -- 172.21.15.26:0/1858672463 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac015070 con 0x56166a07c6a0 2024-04-26T21:10:07.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.700+0000 7f86baa00640 1 -- 172.21.15.26:0/1858672463 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86ac0113f0 con 0x56166a07c6a0 2024-04-26T21:10:07.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.700+0000 7f86baa00640 1 -- 172.21.15.26:0/1858672463 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0117e0 con 0x56166a07c6a0 2024-04-26T21:10:07.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.700+0000 7f86c4571f40 1 -- 172.21.15.26:0/1858672463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56166a114c60 con 0x56166a07c6a0 2024-04-26T21:10:07.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.700+0000 7f86c4571f40 1 -- 172.21.15.26:0/1858672463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56166a06c780 con 0x56166a07c6a0 2024-04-26T21:10:07.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.702+0000 7f86baa00640 1 -- 172.21.15.26:0/1858672463 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86ac018070 con 0x56166a07c6a0 2024-04-26T21:10:07.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.703+0000 7f86baa00640 1 --2- 172.21.15.26:0/1858672463 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f86a006b2d0 0x7f86a006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.703+0000 7f86baa00640 1 -- 172.21.15.26:0/1858672463 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f86ac08a830 con 0x56166a07c6a0 2024-04-26T21:10:07.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.703+0000 7f86c1800640 1 --2- 172.21.15.26:0/1858672463 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f86a006b2d0 0x7f86a006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.703+0000 7f86c4571f40 1 --2- 172.21.15.26:0/1858672463 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x56166a1107e0 0x56166a110c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.703+0000 7f86c4571f40 1 -- 172.21.15.26:0/1858672463 --> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] -- osd_op(unknown.0.0:1 5.13 5:ca8fdbba:::100000001fb.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37) -- 0x56166a111170 con 0x56166a1107e0 2024-04-26T21:10:07.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.704+0000 7f86c2200640 1 --2- 172.21.15.26:0/1858672463 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x56166a1107e0 0x56166a110c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.704+0000 7f86c1800640 1 --2- 172.21.15.26:0/1858672463 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f86a006b2d0 0x7f86a006d790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f86b4050fd0 tx=0x7f86b4051000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:07.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.704+0000 7f86c2200640 1 --2- 172.21.15.26:0/1858672463 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x56166a1107e0 0x56166a110c30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:07.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.705+0000 7f86c2200640 1 -- 172.21.15.26:0/1858672463 <== osd.3 v2:172.21.15.26:6809/1280758061 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (103) out=103b] v0'0 uv3 ondisk = 0) ==== 164+0+103 (crc 0 0 0) 0x7f86bc009040 con 0x56166a1107e0 2024-04-26T21:10:07.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.705+0000 7f86c4571f40 1 -- 172.21.15.26:0/1858672463 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x56166a1107e0 msgr2=0x56166a110c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.705+0000 7f86c4571f40 1 --2- 172.21.15.26:0/1858672463 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x56166a1107e0 0x56166a110c30 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.705+0000 7f86c4571f40 1 -- 172.21.15.26:0/1858672463 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f86a006b2d0 msgr2=0x7f86a006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.705+0000 7f86c4571f40 1 --2- 172.21.15.26:0/1858672463 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f86a006b2d0 0x7f86a006d790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f86b4050fd0 tx=0x7f86b4051000 comp rx=0 tx=0).stop 2024-04-26T21:10:07.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.705+0000 7f86c4571f40 1 -- 172.21.15.26:0/1858672463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166a07c6a0 msgr2=0x56166a06b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.705+0000 7f86c4571f40 1 --2- 172.21.15.26:0/1858672463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56166a07c6a0 0x56166a06b920 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f86ac002fc0 tx=0x7f86ac00c5c0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.705+0000 7f86c2200640 1 -- 172.21.15.26:0/1858672463 reap_dead start 2024-04-26T21:10:07.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.705+0000 7f86c4571f40 1 -- 172.21.15.26:0/1858672463 shutdown_connections 2024-04-26T21:10:07.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.705+0000 7f86c4571f40 1 -- 172.21.15.26:0/1858672463 >> 172.21.15.26:0/1858672463 conn(0x561669fd2100 msgr2=0x56166a07bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:07.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.705+0000 7f86c4571f40 1 -- 172.21.15.26:0/1858672463 shutdown_connections 2024-04-26T21:10:07.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.705+0000 7f86c4571f40 1 -- 172.21.15.26:0/1858672463 wait complete. 2024-04-26T21:10:07.712 INFO:teuthology.orchestra.run.smithi026.stdout:a��alpha parent_b  2024-04-26T21:10:07.712 DEBUG:teuthology.orchestra.run.smithi026:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-26T21:10:07.808 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-26T21:10:07.808 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628283, 2024-04-26T21:10:07.808 INFO:teuthology.orchestra.run.smithi026.stdout: "ancestors": [ 2024-04-26T21:10:07.808 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:10:07.809 INFO:teuthology.orchestra.run.smithi026.stdout: "dirino": 1099511628284, 2024-04-26T21:10:07.809 INFO:teuthology.orchestra.run.smithi026.stdout: "dname": "alpha", 2024-04-26T21:10:07.809 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 9 2024-04-26T21:10:07.809 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:10:07.809 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:10:07.809 INFO:teuthology.orchestra.run.smithi026.stdout: "dirino": 1, 2024-04-26T21:10:07.809 INFO:teuthology.orchestra.run.smithi026.stdout: "dname": "parent_b", 2024-04-26T21:10:07.809 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 32 2024-04-26T21:10:07.809 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:10:07.809 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-26T21:10:07.809 INFO:teuthology.orchestra.run.smithi026.stdout: "pool": 6, 2024-04-26T21:10:07.809 INFO:teuthology.orchestra.run.smithi026.stdout: "old_pools": [ 2024-04-26T21:10:07.809 INFO:teuthology.orchestra.run.smithi026.stdout: 5 2024-04-26T21:10:07.809 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-26T21:10:07.809 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-26T21:10:07.810 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:07.814 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool data_new getxattr 100000001fb.00000000 parent 2024-04-26T21:10:07.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.910+0000 7fa95dfdbf40 1 Processor -- start 2024-04-26T21:10:07.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.910+0000 7fa95dfdbf40 1 -- start start 2024-04-26T21:10:07.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.911+0000 7fa95dfdbf40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556bd90fef50 0x556bd90b2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.911+0000 7fa95dfdbf40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bd918c050 0x556bd91003a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.911+0000 7fa95dfdbf40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd918eeb0 0x556bd90e2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.911+0000 7fa95dfdbf40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556bd9101e70 con 0x556bd918c050 2024-04-26T21:10:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.911+0000 7fa95dfdbf40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556bd905bd20 con 0x556bd918eeb0 2024-04-26T21:10:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.911+0000 7fa95dfdbf40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x556bd918c600 con 0x556bd90fef50 2024-04-26T21:10:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.911+0000 7fa95a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bd918c050 0x556bd91003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.911+0000 7fa95a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bd918c050 0x556bd91003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47672/0 (socket says 172.21.15.26:47672) 2024-04-26T21:10:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.911+0000 7fa95a800640 1 -- 172.21.15.26:0/894641159 learned_addr learned my addr 172.21.15.26:0/894641159 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.911+0000 7fa95bc00640 1 --2- 172.21.15.26:0/894641159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd918eeb0 0x556bd90e2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.911+0000 7fa95b200640 1 --2- 172.21.15.26:0/894641159 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556bd90fef50 0x556bd90b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.912+0000 7fa95a800640 1 -- 172.21.15.26:0/894641159 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556bd90fef50 msgr2=0x556bd90b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.912+0000 7fa95a800640 1 --2- 172.21.15.26:0/894641159 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556bd90fef50 0x556bd90b2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.912+0000 7fa95a800640 1 -- 172.21.15.26:0/894641159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd918eeb0 msgr2=0x556bd90e2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:07.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.912+0000 7fa95a800640 1 --2- 172.21.15.26:0/894641159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd918eeb0 0x556bd90e2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.912+0000 7fa95a800640 1 -- 172.21.15.26:0/894641159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bd9185ce0 con 0x556bd918c050 2024-04-26T21:10:07.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.912+0000 7fa95a800640 1 --2- 172.21.15.26:0/894641159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bd918c050 0x556bd91003a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa94400a5e0 tx=0x7fa944005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f25aea4697c4ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:07.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.912+0000 7fa959400640 1 -- 172.21.15.26:0/894641159 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944008a40 con 0x556bd918c050 2024-04-26T21:10:07.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.912+0000 7fa959400640 1 -- 172.21.15.26:0/894641159 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa944008c00 con 0x556bd918c050 2024-04-26T21:10:07.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.912+0000 7fa959400640 1 -- 172.21.15.26:0/894641159 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944013440 con 0x556bd918c050 2024-04-26T21:10:07.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.913+0000 7fa95dfdbf40 1 -- 172.21.15.26:0/894641159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bd918c050 msgr2=0x556bd91003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.913+0000 7fa95dfdbf40 1 --2- 172.21.15.26:0/894641159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bd918c050 0x556bd91003a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa94400a5e0 tx=0x7fa944005f90 comp rx=0 tx=0).stop 2024-04-26T21:10:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.913+0000 7fa95dfdbf40 1 -- 172.21.15.26:0/894641159 shutdown_connections 2024-04-26T21:10:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.913+0000 7fa95dfdbf40 1 --2- 172.21.15.26:0/894641159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd918eeb0 0x556bd90e2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.913+0000 7fa95dfdbf40 1 --2- 172.21.15.26:0/894641159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bd918c050 0x556bd91003a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.913+0000 7fa95dfdbf40 1 --2- 172.21.15.26:0/894641159 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556bd90fef50 0x556bd90b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.913+0000 7fa95dfdbf40 1 -- 172.21.15.26:0/894641159 >> 172.21.15.26:0/894641159 conn(0x556bd90e4100 msgr2=0x556bd91013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.913+0000 7fa95dfdbf40 1 -- 172.21.15.26:0/894641159 shutdown_connections 2024-04-26T21:10:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.913+0000 7fa95dfdbf40 1 -- 172.21.15.26:0/894641159 wait complete. 2024-04-26T21:10:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.914+0000 7fa95dfdbf40 1 Processor -- start 2024-04-26T21:10:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.914+0000 7fa95dfdbf40 1 -- start start 2024-04-26T21:10:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.915+0000 7fa95dfdbf40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bd90fef50 0x556bd90e7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.915+0000 7fa95dfdbf40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556bd918c050 0x556bd91004c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.915+0000 7fa95dfdbf40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd918eeb0 0x556bd9100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.915+0000 7fa95dfdbf40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556bd90fbe70 con 0x556bd90fef50 2024-04-26T21:10:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.915+0000 7fa95dfdbf40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556bd90b3b90 con 0x556bd918eeb0 2024-04-26T21:10:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.915+0000 7fa95dfdbf40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x556bd90b3f10 con 0x556bd918c050 2024-04-26T21:10:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.915+0000 7fa95b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bd90fef50 0x556bd90e7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.915+0000 7fa95b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bd90fef50 0x556bd90e7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47676/0 (socket says 172.21.15.26:47676) 2024-04-26T21:10:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.915+0000 7fa95b200640 1 -- 172.21.15.26:0/895729058 learned_addr learned my addr 172.21.15.26:0/895729058 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.915+0000 7fa95bc00640 1 --2- 172.21.15.26:0/895729058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd918eeb0 0x556bd9100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.915+0000 7fa95a800640 1 --2- 172.21.15.26:0/895729058 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556bd918c050 0x556bd91004c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.915+0000 7fa95b200640 1 -- 172.21.15.26:0/895729058 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556bd918c050 msgr2=0x556bd91004c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.916+0000 7fa95b200640 1 --2- 172.21.15.26:0/895729058 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x556bd918c050 0x556bd91004c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.916+0000 7fa95b200640 1 -- 172.21.15.26:0/895729058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd918eeb0 msgr2=0x556bd9100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.916+0000 7fa95b200640 1 --2- 172.21.15.26:0/895729058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd918eeb0 0x556bd9100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.916+0000 7fa95b200640 1 -- 172.21.15.26:0/895729058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bd9225520 con 0x556bd90fef50 2024-04-26T21:10:07.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.916+0000 7fa95bc00640 1 --2- 172.21.15.26:0/895729058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556bd918eeb0 0x556bd9100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:07.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.916+0000 7fa95b200640 1 --2- 172.21.15.26:0/895729058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bd90fef50 0x556bd90e7990 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa94c025a80 tx=0x7fa94c003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:07.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.916+0000 7fa953e00640 1 -- 172.21.15.26:0/895729058 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c06a070 con 0x556bd90fef50 2024-04-26T21:10:07.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.916+0000 7fa953e00640 1 -- 172.21.15.26:0/895729058 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa94c05a450 con 0x556bd90fef50 2024-04-26T21:10:07.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.916+0000 7fa953e00640 1 -- 172.21.15.26:0/895729058 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c055040 con 0x556bd90fef50 2024-04-26T21:10:07.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.916+0000 7fa95dfdbf40 1 -- 172.21.15.26:0/895729058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556bd92257c0 con 0x556bd90fef50 2024-04-26T21:10:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.916+0000 7fa95dfdbf40 1 -- 172.21.15.26:0/895729058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bd9225d80 con 0x556bd90fef50 2024-04-26T21:10:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.918+0000 7fa953e00640 1 -- 172.21.15.26:0/895729058 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa94c06d070 con 0x556bd90fef50 2024-04-26T21:10:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.919+0000 7fa953e00640 1 --2- 172.21.15.26:0/895729058 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa93806b300 0x7fa93806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.919+0000 7fa953e00640 1 -- 172.21.15.26:0/895729058 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7fa94c0ce8d0 con 0x556bd90fef50 2024-04-26T21:10:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.920+0000 7fa95a800640 1 --2- 172.21.15.26:0/895729058 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa93806b300 0x7fa93806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.920+0000 7fa95dfdbf40 1 --2- 172.21.15.26:0/895729058 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x556bd9101410 0x556bd92219d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.920+0000 7fa95dfdbf40 1 -- 172.21.15.26:0/895729058 --> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] -- osd_op(unknown.0.0:1 6.13 6:ca8fdbba:::100000001fb.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37) -- 0x556bd9221f10 con 0x556bd9101410 2024-04-26T21:10:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.920+0000 7fa95bc00640 1 --2- 172.21.15.26:0/895729058 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x556bd9101410 0x556bd92219d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.920+0000 7fa95a800640 1 --2- 172.21.15.26:0/895729058 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa93806b300 0x7fa93806d7c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa94400a5e0 tx=0x7fa944015040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.921+0000 7fa95bc00640 1 --2- 172.21.15.26:0/895729058 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x556bd9101410 0x556bd92219d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.921+0000 7fa95bc00640 1 -- 172.21.15.26:0/895729058 <== osd.3 v2:172.21.15.26:6809/1280758061 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (103) out=103b] v0'0 uv1 ondisk = 0) ==== 164+0+103 (crc 0 0 0) 0x7fa95400e450 con 0x556bd9101410 2024-04-26T21:10:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.921+0000 7fa95dfdbf40 1 -- 172.21.15.26:0/895729058 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x556bd9101410 msgr2=0x556bd92219d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.921+0000 7fa95dfdbf40 1 --2- 172.21.15.26:0/895729058 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x556bd9101410 0x556bd92219d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.921+0000 7fa95dfdbf40 1 -- 172.21.15.26:0/895729058 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa93806b300 msgr2=0x7fa93806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.921+0000 7fa95dfdbf40 1 --2- 172.21.15.26:0/895729058 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa93806b300 0x7fa93806d7c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa94400a5e0 tx=0x7fa944015040 comp rx=0 tx=0).stop 2024-04-26T21:10:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.921+0000 7fa95dfdbf40 1 -- 172.21.15.26:0/895729058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bd90fef50 msgr2=0x556bd90e7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.921+0000 7fa95dfdbf40 1 --2- 172.21.15.26:0/895729058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556bd90fef50 0x556bd90e7990 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa94c025a80 tx=0x7fa94c003330 comp rx=0 tx=0).stop 2024-04-26T21:10:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.921+0000 7fa95bc00640 1 -- 172.21.15.26:0/895729058 reap_dead start 2024-04-26T21:10:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.921+0000 7fa95dfdbf40 1 -- 172.21.15.26:0/895729058 shutdown_connections 2024-04-26T21:10:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.921+0000 7fa95dfdbf40 1 -- 172.21.15.26:0/895729058 >> 172.21.15.26:0/895729058 conn(0x556bd90e4100 msgr2=0x556bd90b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.921+0000 7fa95dfdbf40 1 -- 172.21.15.26:0/895729058 shutdown_connections 2024-04-26T21:10:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:07.921+0000 7fa95dfdbf40 1 -- 172.21.15.26:0/895729058 wait complete. 2024-04-26T21:10:07.927 INFO:teuthology.orchestra.run.smithi026.stdout:a��alpha parent_b  2024-04-26T21:10:07.928 DEBUG:teuthology.orchestra.run.smithi026:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-26T21:10:08.033 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-26T21:10:08.033 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628283, 2024-04-26T21:10:08.033 INFO:teuthology.orchestra.run.smithi026.stdout: "ancestors": [ 2024-04-26T21:10:08.034 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:10:08.034 INFO:teuthology.orchestra.run.smithi026.stdout: "dirino": 1099511628284, 2024-04-26T21:10:08.034 INFO:teuthology.orchestra.run.smithi026.stdout: "dname": "alpha", 2024-04-26T21:10:08.034 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 9 2024-04-26T21:10:08.034 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:10:08.034 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:10:08.034 INFO:teuthology.orchestra.run.smithi026.stdout: "dirino": 1, 2024-04-26T21:10:08.034 INFO:teuthology.orchestra.run.smithi026.stdout: "dname": "parent_b", 2024-04-26T21:10:08.034 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 32 2024-04-26T21:10:08.034 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:10:08.034 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-26T21:10:08.034 INFO:teuthology.orchestra.run.smithi026.stdout: "pool": 6, 2024-04-26T21:10:08.034 INFO:teuthology.orchestra.run.smithi026.stdout: "old_pools": [ 2024-04-26T21:10:08.035 INFO:teuthology.orchestra.run.smithi026.stdout: 5 2024-04-26T21:10:08.035 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-26T21:10:08.035 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-26T21:10:08.035 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:08.039 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool data_new getxattr 100000001fb.00000000 layout 2024-04-26T21:10:08.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.146+0000 7f2721140f40 1 Processor -- start 2024-04-26T21:10:08.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.146+0000 7f2721140f40 1 -- start start 2024-04-26T21:10:08.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.147+0000 7f2721140f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55630287cf50 0x556302865990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:08.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.147+0000 7f2721140f40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55630290a050 0x55630287f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:08.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.147+0000 7f2721140f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630290ceb0 0x556302830570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:08.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.147+0000 7f2721140f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55630287d900 con 0x55630287cf50 2024-04-26T21:10:08.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.147+0000 7f2721140f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5563027d9d20 con 0x55630290ceb0 2024-04-26T21:10:08.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.147+0000 7f2721140f40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55630290a600 con 0x55630290a050 2024-04-26T21:10:08.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.147+0000 7f271e400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55630287cf50 0x556302865990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:08.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.147+0000 7f271e400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55630287cf50 0x556302865990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47682/0 (socket says 172.21.15.26:47682) 2024-04-26T21:10:08.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.147+0000 7f271e400640 1 -- 172.21.15.26:0/1706570643 learned_addr learned my addr 172.21.15.26:0/1706570643 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:08.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.148+0000 7f271ee00640 1 --2- 172.21.15.26:0/1706570643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630290ceb0 0x556302830570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:08.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.148+0000 7f271da00640 1 --2- 172.21.15.26:0/1706570643 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55630290a050 0x55630287f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:08.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.148+0000 7f271e400640 1 -- 172.21.15.26:0/1706570643 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55630290a050 msgr2=0x55630287f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:08.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.148+0000 7f271e400640 1 --2- 172.21.15.26:0/1706570643 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55630290a050 0x55630287f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:08.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.148+0000 7f271e400640 1 -- 172.21.15.26:0/1706570643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630290ceb0 msgr2=0x556302830570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:08.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.148+0000 7f271e400640 1 --2- 172.21.15.26:0/1706570643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630290ceb0 0x556302830570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:08.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.148+0000 7f271e400640 1 -- 172.21.15.26:0/1706570643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55630287a100 con 0x55630287cf50 2024-04-26T21:10:08.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.148+0000 7f271e400640 1 --2- 172.21.15.26:0/1706570643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55630287cf50 0x556302865990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2710000f80 tx=0x7f2710050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49585f36ec45fcbb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:08.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.148+0000 7f2717e00640 1 -- 172.21.15.26:0/1706570643 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710055030 con 0x55630287cf50 2024-04-26T21:10:08.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.149+0000 7f2717e00640 1 -- 172.21.15.26:0/1706570643 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2710051bb0 con 0x55630287cf50 2024-04-26T21:10:08.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.149+0000 7f2717e00640 1 -- 172.21.15.26:0/1706570643 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271005b430 con 0x55630287cf50 2024-04-26T21:10:08.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.149+0000 7f2721140f40 1 -- 172.21.15.26:0/1706570643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55630287cf50 msgr2=0x556302865990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:08.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.149+0000 7f2721140f40 1 --2- 172.21.15.26:0/1706570643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55630287cf50 0x556302865990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2710000f80 tx=0x7f2710050000 comp rx=0 tx=0).stop 2024-04-26T21:10:08.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.149+0000 7f2721140f40 1 -- 172.21.15.26:0/1706570643 shutdown_connections 2024-04-26T21:10:08.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.149+0000 7f2721140f40 1 --2- 172.21.15.26:0/1706570643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630290ceb0 0x556302830570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:08.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.149+0000 7f2721140f40 1 --2- 172.21.15.26:0/1706570643 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55630290a050 0x55630287f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:08.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.149+0000 7f2721140f40 1 --2- 172.21.15.26:0/1706570643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55630287cf50 0x556302865990 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:08.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.149+0000 7f2721140f40 1 -- 172.21.15.26:0/1706570643 >> 172.21.15.26:0/1706570643 conn(0x556302862100 msgr2=0x556302880780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:08.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.149+0000 7f2721140f40 1 -- 172.21.15.26:0/1706570643 shutdown_connections 2024-04-26T21:10:08.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.149+0000 7f2721140f40 1 -- 172.21.15.26:0/1706570643 wait complete. 2024-04-26T21:10:08.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.150+0000 7f2721140f40 1 Processor -- start 2024-04-26T21:10:08.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.150+0000 7f2721140f40 1 -- start start 2024-04-26T21:10:08.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.151+0000 7f2721140f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55630287cf50 0x5563028655c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:08.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.151+0000 7f2721140f40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55630290a050 0x556302902f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:08.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.151+0000 7f2721140f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630290ceb0 0x5563028806e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:08.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.151+0000 7f2721140f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556302879c70 con 0x55630287cf50 2024-04-26T21:10:08.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.151+0000 7f2721140f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55630290ccf0 con 0x55630290ceb0 2024-04-26T21:10:08.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.151+0000 7f2721140f40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x556302831f50 con 0x55630290a050 2024-04-26T21:10:08.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.152+0000 7f271e400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55630287cf50 0x5563028655c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:08.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.152+0000 7f271e400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55630287cf50 0x5563028655c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47690/0 (socket says 172.21.15.26:47690) 2024-04-26T21:10:08.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.152+0000 7f271e400640 1 -- 172.21.15.26:0/2664901554 learned_addr learned my addr 172.21.15.26:0/2664901554 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:08.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.152+0000 7f271da00640 1 --2- 172.21.15.26:0/2664901554 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55630290a050 0x556302902f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:08.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.152+0000 7f271ee00640 1 --2- 172.21.15.26:0/2664901554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630290ceb0 0x5563028806e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:08.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.152+0000 7f271e400640 1 -- 172.21.15.26:0/2664901554 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55630290a050 msgr2=0x556302902f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:08.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.152+0000 7f271e400640 1 --2- 172.21.15.26:0/2664901554 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55630290a050 0x556302902f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:08.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.152+0000 7f271e400640 1 -- 172.21.15.26:0/2664901554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630290ceb0 msgr2=0x5563028806e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:08.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.152+0000 7f271e400640 1 --2- 172.21.15.26:0/2664901554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630290ceb0 0x5563028806e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:08.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.152+0000 7f271e400640 1 -- 172.21.15.26:0/2664901554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556302831620 con 0x55630287cf50 2024-04-26T21:10:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.152+0000 7f271da00640 1 --2- 172.21.15.26:0/2664901554 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55630290a050 0x556302902f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.152+0000 7f271ee00640 1 --2- 172.21.15.26:0/2664901554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55630290ceb0 0x5563028806e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.152+0000 7f271e400640 1 --2- 172.21.15.26:0/2664901554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55630287cf50 0x5563028655c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2710025ab0 tx=0x7f2710003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.153+0000 7f2716a00640 1 -- 172.21.15.26:0/2664901554 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710074070 con 0x55630287cf50 2024-04-26T21:10:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.153+0000 7f2716a00640 1 -- 172.21.15.26:0/2664901554 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27100519b0 con 0x55630287cf50 2024-04-26T21:10:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.153+0000 7f2716a00640 1 -- 172.21.15.26:0/2664901554 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710055040 con 0x55630287cf50 2024-04-26T21:10:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.153+0000 7f2721140f40 1 -- 172.21.15.26:0/2664901554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556302831900 con 0x55630287cf50 2024-04-26T21:10:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.153+0000 7f2721140f40 1 -- 172.21.15.26:0/2664901554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556302903440 con 0x55630287cf50 2024-04-26T21:10:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.155+0000 7f2716a00640 1 -- 172.21.15.26:0/2664901554 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2710058020 con 0x55630287cf50 2024-04-26T21:10:08.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.156+0000 7f2716a00640 1 --2- 172.21.15.26:0/2664901554 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f26fc06b2d0 0x7f26fc06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:08.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.156+0000 7f2716a00640 1 -- 172.21.15.26:0/2664901554 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f27100cfa30 con 0x55630287cf50 2024-04-26T21:10:08.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.156+0000 7f2721140f40 1 --2- 172.21.15.26:0/2664901554 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x556302880c20 0x5563028323e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:08.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.156+0000 7f271da00640 1 --2- 172.21.15.26:0/2664901554 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f26fc06b2d0 0x7f26fc06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:08.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.156+0000 7f2721140f40 1 -- 172.21.15.26:0/2664901554 --> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] -- osd_op(unknown.0.0:1 6.13 6:ca8fdbba:::100000001fb.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37) -- 0x556302832920 con 0x556302880c20 2024-04-26T21:10:08.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.157+0000 7f271ee00640 1 --2- 172.21.15.26:0/2664901554 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x556302880c20 0x5563028323e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:08.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.157+0000 7f271da00640 1 --2- 172.21.15.26:0/2664901554 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f26fc06b2d0 0x7f26fc06d790 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55630285f900 tx=0x7f27080025b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:08.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.157+0000 7f271ee00640 1 --2- 172.21.15.26:0/2664901554 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x556302880c20 0x5563028323e0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:08.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.158+0000 7f271ee00640 1 -- 172.21.15.26:0/2664901554 <== osd.3 v2:172.21.15.26:6809/1280758061 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (30) out=30b] v0'0 uv1 ondisk = 0) ==== 164+0+30 (crc 0 0 0) 0x7f2718009a00 con 0x556302880c20 2024-04-26T21:10:08.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.158+0000 7f2721140f40 1 -- 172.21.15.26:0/2664901554 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x556302880c20 msgr2=0x5563028323e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:08.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.158+0000 7f2721140f40 1 --2- 172.21.15.26:0/2664901554 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x556302880c20 0x5563028323e0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:08.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.158+0000 7f2721140f40 1 -- 172.21.15.26:0/2664901554 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f26fc06b2d0 msgr2=0x7f26fc06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:08.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.158+0000 7f2721140f40 1 --2- 172.21.15.26:0/2664901554 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f26fc06b2d0 0x7f26fc06d790 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55630285f900 tx=0x7f27080025b0 comp rx=0 tx=0).stop 2024-04-26T21:10:08.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.158+0000 7f2721140f40 1 -- 172.21.15.26:0/2664901554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55630287cf50 msgr2=0x5563028655c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:08.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.158+0000 7f2721140f40 1 --2- 172.21.15.26:0/2664901554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55630287cf50 0x5563028655c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2710025ab0 tx=0x7f2710003330 comp rx=0 tx=0).stop 2024-04-26T21:10:08.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.158+0000 7f271ee00640 1 -- 172.21.15.26:0/2664901554 reap_dead start 2024-04-26T21:10:08.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.158+0000 7f2721140f40 1 -- 172.21.15.26:0/2664901554 shutdown_connections 2024-04-26T21:10:08.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.158+0000 7f2721140f40 1 -- 172.21.15.26:0/2664901554 >> 172.21.15.26:0/2664901554 conn(0x556302862100 msgr2=0x556302879480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:08.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.158+0000 7f2721140f40 1 -- 172.21.15.26:0/2664901554 shutdown_connections 2024-04-26T21:10:08.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.158+0000 7f2721140f40 1 -- 172.21.15.26:0/2664901554 wait complete. 2024-04-26T21:10:08.165 INFO:teuthology.orchestra.run.smithi026.stdout:@@ 2024-04-26T21:10:08.165 DEBUG:teuthology.orchestra.run.smithi026:> ceph-dencoder type file_layout_t import - decode dump_json 2024-04-26T21:10:08.260 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-26T21:10:08.260 INFO:teuthology.orchestra.run.smithi026.stdout: "stripe_unit": 4194304, 2024-04-26T21:10:08.260 INFO:teuthology.orchestra.run.smithi026.stdout: "stripe_count": 1, 2024-04-26T21:10:08.260 INFO:teuthology.orchestra.run.smithi026.stdout: "object_size": 4194304, 2024-04-26T21:10:08.260 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_id": 6, 2024-04-26T21:10:08.261 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_ns": "" 2024-04-26T21:10:08.261 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-26T21:10:08.261 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:08.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:10:08.266 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_c) 2024-04-26T21:10:08.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:10:08.323 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mv parent_b/alpha parent_c/alpha) 2024-04-26T21:10:08.380 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:10:08.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.588+0000 7fc15be00640 1 Processor -- start 2024-04-26T21:10:08.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.588+0000 7fc15be00640 1 -- start start 2024-04-26T21:10:08.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.590+0000 7fc15be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc15c06f340 0x7fc15c15faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:08.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.590+0000 7fc15be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc15c15ffe0 0x7fc15c0d11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:08.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.590+0000 7fc15be00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc15c06f7d0 con 0x7fc15c073470 2024-04-26T21:10:08.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.590+0000 7fc15be00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc15c073f60 con 0x7fc15c15ffe0 2024-04-26T21:10:08.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.590+0000 7fc15be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc15c06fab0 con 0x7fc15c06f340 2024-04-26T21:10:08.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.590+0000 7fc15aa00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc15c073470 0x7fc15c073850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55508/0 (socket says 172.21.15.26:55508) 2024-04-26T21:10:08.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.590+0000 7fc15aa00640 1 -- 172.21.15.26:0/267895068 learned_addr learned my addr 172.21.15.26:0/267895068 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:08.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.591+0000 7fc15a000640 1 --2- 172.21.15.26:0/267895068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc15c06f340 0x7fc15c15faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:08.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.591+0000 7fc15b400640 1 --2- 172.21.15.26:0/267895068 >> v2:172.21.15.86:3300/0 conn(0x7fc15c15ffe0 0x7fc15c0d11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:08.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.591+0000 7fc159600640 1 -- 172.21.15.26:0/267895068 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1041683132 0 0) 0x7fc15c06f7d0 con 0x7fc15c073470 2024-04-26T21:10:08.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.591+0000 7fc159600640 1 -- 172.21.15.26:0/267895068 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc140005730 con 0x7fc15c073470 2024-04-26T21:10:08.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.591+0000 7fc159600640 1 -- 172.21.15.26:0/267895068 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 709806429 0 0) 0x7fc140005730 con 0x7fc15c073470 2024-04-26T21:10:08.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.591+0000 7fc159600640 1 -- 172.21.15.26:0/267895068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc15c06f340 msgr2=0x7fc15c15faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:08.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.591+0000 7fc159600640 1 --2- 172.21.15.26:0/267895068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc15c06f340 0x7fc15c15faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:08.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.591+0000 7fc159600640 1 -- 172.21.15.26:0/267895068 >> v2:172.21.15.86:3300/0 conn(0x7fc15c15ffe0 msgr2=0x7fc15c0d11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:08.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.591+0000 7fc159600640 1 --2- 172.21.15.26:0/267895068 >> v2:172.21.15.86:3300/0 conn(0x7fc15c15ffe0 0x7fc15c0d11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:08.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.591+0000 7fc159600640 1 -- 172.21.15.26:0/267895068 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15c06fd90 con 0x7fc15c073470 2024-04-26T21:10:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.591+0000 7fc159600640 1 -- 172.21.15.26:0/267895068 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7fc144001670 con 0x7fc15c073470 2024-04-26T21:10:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.592+0000 7fc159600640 1 -- 172.21.15.26:0/267895068 >> v1:172.21.15.26:6789/0 conn(0x7fc15c073470 legacy=0x7fc15c073850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.592+0000 7fc159600640 1 --2- 172.21.15.26:0/267895068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc140005bc0 0x7fc140005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.592+0000 7fc159600640 1 --2- 172.21.15.26:0/267895068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc140006a10 0x7fc140006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.592+0000 7fc159600640 1 --2- 172.21.15.26:0/267895068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc140007390 0x7fc14000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.592+0000 7fc159600640 1 -- 172.21.15.26:0/267895068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c073f60 con 0x7fc140005bc0 2024-04-26T21:10:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.592+0000 7fc159600640 1 -- 172.21.15.26:0/267895068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c06fab0 con 0x7fc140007390 2024-04-26T21:10:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.592+0000 7fc159600640 1 -- 172.21.15.26:0/267895068 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc14000e110 con 0x7fc140006a10 2024-04-26T21:10:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.592+0000 7fc15aa00640 1 --2- 172.21.15.26:0/267895068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc140005bc0 0x7fc140005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.592+0000 7fc15b400640 1 --2- 172.21.15.26:0/267895068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc140007390 0x7fc14000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.592+0000 7fc15a000640 1 --2- 172.21.15.26:0/267895068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc140006a10 0x7fc140006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.592+0000 7fc15aa00640 1 -- 172.21.15.26:0/267895068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc140006a10 msgr2=0x7fc140006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.592+0000 7fc15aa00640 1 --2- 172.21.15.26:0/267895068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc140006a10 0x7fc140006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.592+0000 7fc15aa00640 1 -- 172.21.15.26:0/267895068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc140007390 msgr2=0x7fc14000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:08.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.592+0000 7fc15aa00640 1 --2- 172.21.15.26:0/267895068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc140007390 0x7fc14000dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:08.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.592+0000 7fc15aa00640 1 -- 172.21.15.26:0/267895068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc140005730 con 0x7fc140005bc0 2024-04-26T21:10:08.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.593+0000 7fc15b400640 1 --2- 172.21.15.26:0/267895068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc140007390 0x7fc14000dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:08.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.593+0000 7fc15a000640 1 --2- 172.21.15.26:0/267895068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc140006a10 0x7fc140006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:08.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.594+0000 7fc15aa00640 1 --2- 172.21.15.26:0/267895068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc140005bc0 0x7fc140005fe0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc14400c190 tx=0x7fc140007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ae0a8331b0af7b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.594+0000 7fc159600640 1 -- 172.21.15.26:0/267895068 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc144007c60 con 0x7fc140005bc0 2024-04-26T21:10:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.594+0000 7fc159600640 1 -- 172.21.15.26:0/267895068 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc144005af0 con 0x7fc140005bc0 2024-04-26T21:10:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.594+0000 7fc159600640 1 -- 172.21.15.26:0/267895068 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc144007070 con 0x7fc140005bc0 2024-04-26T21:10:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.594+0000 7fc15be00640 1 -- 172.21.15.26:0/267895068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc140005bc0 msgr2=0x7fc140005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.594+0000 7fc15be00640 1 --2- 172.21.15.26:0/267895068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc140005bc0 0x7fc140005fe0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc14400c190 tx=0x7fc140007a40 comp rx=0 tx=0).stop 2024-04-26T21:10:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.595+0000 7fc15b400640 1 -- 172.21.15.26:0/267895068 reap_dead start 2024-04-26T21:10:08.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.595+0000 7fc15be00640 1 -- 172.21.15.26:0/267895068 shutdown_connections 2024-04-26T21:10:08.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.595+0000 7fc15be00640 1 -- 172.21.15.26:0/267895068 >> 172.21.15.26:0/267895068 conn(0x7fc15c06c690 msgr2=0x7fc15c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:08.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.595+0000 7fc15be00640 1 -- 172.21.15.26:0/267895068 shutdown_connections 2024-04-26T21:10:08.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.595+0000 7fc15be00640 1 -- 172.21.15.26:0/267895068 wait complete. 2024-04-26T21:10:08.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.597+0000 7fc15be00640 1 Processor -- start 2024-04-26T21:10:08.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.597+0000 7fc15be00640 1 -- start start 2024-04-26T21:10:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.598+0000 7fc15be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc140005bc0 0x7fc15c15db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.598+0000 7fc15be00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc15c0ae850 0x7fc15c0b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.598+0000 7fc15be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc15c14d6e0 0x7fc15c14db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.598+0000 7fc15be00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c0b4d40 con 0x7fc15c14d6e0 2024-04-26T21:10:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.598+0000 7fc15be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c15c4b0 con 0x7fc140005bc0 2024-04-26T21:10:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.598+0000 7fc15be00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc15c0b4ec0 con 0x7fc15c0ae850 2024-04-26T21:10:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.598+0000 7fc15b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc15c14d6e0 0x7fc15c14db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.598+0000 7fc15b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc15c14d6e0 0x7fc15c14db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47716/0 (socket says 172.21.15.26:47716) 2024-04-26T21:10:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.598+0000 7fc15b400640 1 -- 172.21.15.26:0/2894230601 learned_addr learned my addr 172.21.15.26:0/2894230601 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.598+0000 7fc15a000640 1 --2- 172.21.15.26:0/2894230601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc15c0ae850 0x7fc15c0b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.598+0000 7fc15aa00640 1 --2- 172.21.15.26:0/2894230601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc140005bc0 0x7fc15c15db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.599+0000 7fc15b400640 1 -- 172.21.15.26:0/2894230601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc15c0ae850 msgr2=0x7fc15c0b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.599+0000 7fc15b400640 1 --2- 172.21.15.26:0/2894230601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc15c0ae850 0x7fc15c0b22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.599+0000 7fc15b400640 1 -- 172.21.15.26:0/2894230601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc140005bc0 msgr2=0x7fc15c15db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.599+0000 7fc15b400640 1 --2- 172.21.15.26:0/2894230601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc140005bc0 0x7fc15c15db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.599+0000 7fc15b400640 1 -- 172.21.15.26:0/2894230601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15c15e270 con 0x7fc15c14d6e0 2024-04-26T21:10:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.599+0000 7fc15b400640 1 --2- 172.21.15.26:0/2894230601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc15c14d6e0 0x7fc15c14db60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc15c1603e0 tx=0x7fc154010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.599+0000 7fc153400640 1 -- 172.21.15.26:0/2894230601 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15400f040 con 0x7fc15c14d6e0 2024-04-26T21:10:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.599+0000 7fc153400640 1 -- 172.21.15.26:0/2894230601 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc15400e8d0 con 0x7fc15c14d6e0 2024-04-26T21:10:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.599+0000 7fc153400640 1 -- 172.21.15.26:0/2894230601 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15400eca0 con 0x7fc15c14d6e0 2024-04-26T21:10:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.599+0000 7fc15be00640 1 -- 172.21.15.26:0/2894230601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc15c15e550 con 0x7fc15c14d6e0 2024-04-26T21:10:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.600+0000 7fc15be00640 1 -- 172.21.15.26:0/2894230601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc15c15eb10 con 0x7fc15c14d6e0 2024-04-26T21:10:08.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.601+0000 7fc15be00640 1 -- 172.21.15.26:0/2894230601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc15c06a700 con 0x7fc15c14d6e0 2024-04-26T21:10:08.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.602+0000 7fc153400640 1 -- 172.21.15.26:0/2894230601 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1540253f0 con 0x7fc15c14d6e0 2024-04-26T21:10:08.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.603+0000 7fc153400640 1 --2- 172.21.15.26:0/2894230601 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc13806c920 0x7fc13806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:08.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.603+0000 7fc153400640 1 -- 172.21.15.26:0/2894230601 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7fc15408bf70 con 0x7fc15c14d6e0 2024-04-26T21:10:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.603+0000 7fc15aa00640 1 --2- 172.21.15.26:0/2894230601 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc13806c920 0x7fc13806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.604+0000 7fc15aa00640 1 --2- 172.21.15.26:0/2894230601 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc13806c920 0x7fc13806ede0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc144032000 tx=0x7fc144007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:08.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.608+0000 7fc153400640 1 -- 172.21.15.26:0/2894230601 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc154018030 con 0x7fc15c14d6e0 2024-04-26T21:10:08.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.859+0000 7fc15be00640 1 -- 172.21.15.26:0/2894230601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc15c15c280 con 0x7fc15c14d6e0 2024-04-26T21:10:08.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.860+0000 7fc153400640 1 -- 172.21.15.26:0/2894230601 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5180 (secure 0 0 0) 0x7fc154059bb0 con 0x7fc15c14d6e0 2024-04-26T21:10:08.862 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:08.862 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2349160700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2349160700},{"type":"v1","addr":"172.21.15.26:6837","nonce":2349160700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4598,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2824191762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2824191762},{"type":"v1","addr":"172.21.15.86:6834","nonce":2824191762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/263746310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":263746310},{"type":"v1","addr":"172.21.15.86:6835","nonce":263746310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:09:48.622025+0000","modified":"2024-04-26T21:10:05.296141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5058":{"gid":5058,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6836/2528180854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2528180854},{"type":"v1","addr":"172.21.15.26:6836","nonce":2528180854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5058,"qdb_cluster":[5058]},"id":2}]} 2024-04-26T21:10:08.862 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 18 2024-04-26T21:10:08.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.863+0000 7fc15be00640 1 -- 172.21.15.26:0/2894230601 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc13806c920 msgr2=0x7fc13806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:08.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.863+0000 7fc15be00640 1 --2- 172.21.15.26:0/2894230601 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc13806c920 0x7fc13806ede0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc144032000 tx=0x7fc144007000 comp rx=0 tx=0).stop 2024-04-26T21:10:08.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.863+0000 7fc15be00640 1 -- 172.21.15.26:0/2894230601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc15c14d6e0 msgr2=0x7fc15c14db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:08.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.863+0000 7fc15be00640 1 --2- 172.21.15.26:0/2894230601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc15c14d6e0 0x7fc15c14db60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc15c1603e0 tx=0x7fc154010c20 comp rx=0 tx=0).stop 2024-04-26T21:10:08.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.863+0000 7fc15be00640 1 -- 172.21.15.26:0/2894230601 shutdown_connections 2024-04-26T21:10:08.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.863+0000 7fc15be00640 1 --2- 172.21.15.26:0/2894230601 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc13806c920 0x7fc13806ede0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:08.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.863+0000 7fc15be00640 1 --2- 172.21.15.26:0/2894230601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc15c14d6e0 0x7fc15c14db60 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:08.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.863+0000 7fc15be00640 1 --2- 172.21.15.26:0/2894230601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc15c0ae850 0x7fc15c0b22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:08.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.863+0000 7fc15be00640 1 --2- 172.21.15.26:0/2894230601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc140005bc0 0x7fc15c15db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:08.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.863+0000 7fc15be00640 1 -- 172.21.15.26:0/2894230601 >> 172.21.15.26:0/2894230601 conn(0x7fc15c06c690 msgr2=0x7fc15c14f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:08.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.863+0000 7fc15be00640 1 -- 172.21.15.26:0/2894230601 shutdown_connections 2024-04-26T21:10:08.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:08.863+0000 7fc15be00640 1 -- 172.21.15.26:0/2894230601 wait complete. 2024-04-26T21:10:08.885 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-26T21:10:09.136 INFO:teuthology.orchestra.run.smithi026.stdout:{"message":"","return_code":0} 2024-04-26T21:10:09.136 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-26T21:10:09.137 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001fb.00000000 parent 2024-04-26T21:10:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.246+0000 7fa261d1ff40 1 Processor -- start 2024-04-26T21:10:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.246+0000 7fa261d1ff40 1 -- start start 2024-04-26T21:10:09.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.247+0000 7fa261d1ff40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e5d85a8f50 0x55e5d855c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.247+0000 7fa261d1ff40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5d8636050 0x55e5d86371a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.247+0000 7fa261d1ff40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5d8627f70 0x55e5d86376e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.247+0000 7fa261d1ff40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e5d85abe70 con 0x55e5d8627f70 2024-04-26T21:10:09.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.247+0000 7fa261d1ff40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e5d8505d20 con 0x55e5d8636050 2024-04-26T21:10:09.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.247+0000 7fa261d1ff40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55e5d8636600 con 0x55e5d85a8f50 2024-04-26T21:10:09.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.247+0000 7fa25ee00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5d8627f70 0x55e5d86376e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.247+0000 7fa25ee00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5d8627f70 0x55e5d86376e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46760/0 (socket says 172.21.15.26:46760) 2024-04-26T21:10:09.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.247+0000 7fa25ee00640 1 -- 172.21.15.26:0/4111123659 learned_addr learned my addr 172.21.15.26:0/4111123659 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:09.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.248+0000 7fa25e400640 1 --2- 172.21.15.26:0/4111123659 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e5d85a8f50 0x55e5d855c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.248+0000 7fa255a00640 1 --2- 172.21.15.26:0/4111123659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5d8636050 0x55e5d86371a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.248+0000 7fa25ee00640 1 -- 172.21.15.26:0/4111123659 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e5d85a8f50 msgr2=0x55e5d855c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:09.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.248+0000 7fa25ee00640 1 --2- 172.21.15.26:0/4111123659 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e5d85a8f50 0x55e5d855c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.248+0000 7fa25ee00640 1 -- 172.21.15.26:0/4111123659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5d8636050 msgr2=0x55e5d86371a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:09.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.248+0000 7fa25ee00640 1 --2- 172.21.15.26:0/4111123659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5d8636050 0x55e5d86371a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.248+0000 7fa25ee00640 1 -- 172.21.15.26:0/4111123659 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5d85aab30 con 0x55e5d8627f70 2024-04-26T21:10:09.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.248+0000 7fa25ee00640 1 --2- 172.21.15.26:0/4111123659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5d8627f70 0x55e5d86376e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa2580015d0 tx=0x7fa258001a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e288aa66eab76fa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:09.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.248+0000 7fa25d000640 1 -- 172.21.15.26:0/4111123659 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa258004520 con 0x55e5d8627f70 2024-04-26T21:10:09.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.248+0000 7fa25d000640 1 -- 172.21.15.26:0/4111123659 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2580046c0 con 0x55e5d8627f70 2024-04-26T21:10:09.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.248+0000 7fa25d000640 1 -- 172.21.15.26:0/4111123659 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa258004af0 con 0x55e5d8627f70 2024-04-26T21:10:09.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.249+0000 7fa261d1ff40 1 -- 172.21.15.26:0/4111123659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5d8627f70 msgr2=0x55e5d86376e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:09.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.249+0000 7fa261d1ff40 1 --2- 172.21.15.26:0/4111123659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5d8627f70 0x55e5d86376e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa2580015d0 tx=0x7fa258001a90 comp rx=0 tx=0).stop 2024-04-26T21:10:09.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.249+0000 7fa261d1ff40 1 -- 172.21.15.26:0/4111123659 shutdown_connections 2024-04-26T21:10:09.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.249+0000 7fa261d1ff40 1 --2- 172.21.15.26:0/4111123659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5d8627f70 0x55e5d86376e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.249+0000 7fa261d1ff40 1 --2- 172.21.15.26:0/4111123659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5d8636050 0x55e5d86371a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.249+0000 7fa261d1ff40 1 --2- 172.21.15.26:0/4111123659 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e5d85a8f50 0x55e5d855c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.249+0000 7fa261d1ff40 1 -- 172.21.15.26:0/4111123659 >> 172.21.15.26:0/4111123659 conn(0x55e5d858e100 msgr2=0x55e5d85aa380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:09.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.249+0000 7fa261d1ff40 1 -- 172.21.15.26:0/4111123659 shutdown_connections 2024-04-26T21:10:09.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.249+0000 7fa261d1ff40 1 -- 172.21.15.26:0/4111123659 wait complete. 2024-04-26T21:10:09.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.250+0000 7fa261d1ff40 1 Processor -- start 2024-04-26T21:10:09.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.250+0000 7fa261d1ff40 1 -- start start 2024-04-26T21:10:09.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.251+0000 7fa261d1ff40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5d85a8f50 0x55e5d85a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.251+0000 7fa261d1ff40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5d8627f70 0x55e5d858fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.251+0000 7fa261d1ff40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e5d8636050 0x55e5d855c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.251+0000 7fa261d1ff40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e5d855ed90 con 0x55e5d85a8f50 2024-04-26T21:10:09.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.251+0000 7fa261d1ff40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e5d855e620 con 0x55e5d8627f70 2024-04-26T21:10:09.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.251+0000 7fa261d1ff40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55e5d8628bb0 con 0x55e5d8636050 2024-04-26T21:10:09.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.251+0000 7fa25e400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5d85a8f50 0x55e5d85a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.251+0000 7fa25e400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5d85a8f50 0x55e5d85a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46770/0 (socket says 172.21.15.26:46770) 2024-04-26T21:10:09.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.251+0000 7fa25e400640 1 -- 172.21.15.26:0/1927315764 learned_addr learned my addr 172.21.15.26:0/1927315764 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:09.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.252+0000 7fa25ee00640 1 --2- 172.21.15.26:0/1927315764 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e5d8636050 0x55e5d855c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.252+0000 7fa25e400640 1 -- 172.21.15.26:0/1927315764 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e5d8636050 msgr2=0x55e5d855c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:09.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.252+0000 7fa25e400640 1 --2- 172.21.15.26:0/1927315764 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55e5d8636050 0x55e5d855c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.252+0000 7fa25e400640 1 -- 172.21.15.26:0/1927315764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5d8627f70 msgr2=0x55e5d858fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:09.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.252+0000 7fa255a00640 1 --2- 172.21.15.26:0/1927315764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5d8627f70 0x55e5d858fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.252+0000 7fa25e400640 1 --2- 172.21.15.26:0/1927315764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e5d8627f70 0x55e5d858fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.252+0000 7fa25e400640 1 -- 172.21.15.26:0/1927315764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5d862fcc0 con 0x55e5d85a8f50 2024-04-26T21:10:09.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.252+0000 7fa25e400640 1 --2- 172.21.15.26:0/1927315764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5d85a8f50 0x55e5d85a56a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa250025cc0 tx=0x7fa250002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:09.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.253+0000 7fa257400640 1 -- 172.21.15.26:0/1927315764 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa250054a00 con 0x55e5d85a8f50 2024-04-26T21:10:09.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.253+0000 7fa257400640 1 -- 172.21.15.26:0/1927315764 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa250054ba0 con 0x55e5d85a8f50 2024-04-26T21:10:09.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.253+0000 7fa257400640 1 -- 172.21.15.26:0/1927315764 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25005f420 con 0x55e5d85a8f50 2024-04-26T21:10:09.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.253+0000 7fa261d1ff40 1 -- 172.21.15.26:0/1927315764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e5d8626c10 con 0x55e5d85a8f50 2024-04-26T21:10:09.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.253+0000 7fa261d1ff40 1 -- 172.21.15.26:0/1927315764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5d862f9e0 con 0x55e5d85a8f50 2024-04-26T21:10:09.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.255+0000 7fa257400640 1 -- 172.21.15.26:0/1927315764 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa25005c040 con 0x55e5d85a8f50 2024-04-26T21:10:09.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.256+0000 7fa257400640 1 --2- 172.21.15.26:0/1927315764 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa23c06b2d0 0x7fa23c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.256+0000 7fa257400640 1 -- 172.21.15.26:0/1927315764 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7fa2500cec70 con 0x55e5d85a8f50 2024-04-26T21:10:09.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.256+0000 7fa255a00640 1 --2- 172.21.15.26:0/1927315764 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa23c06b2d0 0x7fa23c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.256+0000 7fa261d1ff40 1 --2- 172.21.15.26:0/1927315764 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x55e5d8590340 0x55e5d86273d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.256+0000 7fa261d1ff40 1 -- 172.21.15.26:0/1927315764 --> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] -- osd_op(unknown.0.0:1 5.13 5:ca8fdbba:::100000001fb.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37) -- 0x55e5d855cf20 con 0x55e5d8590340 2024-04-26T21:10:09.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.256+0000 7fa25ee00640 1 --2- 172.21.15.26:0/1927315764 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x55e5d8590340 0x55e5d86273d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.257+0000 7fa255a00640 1 --2- 172.21.15.26:0/1927315764 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa23c06b2d0 0x7fa23c06d790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55e5d8638a20 tx=0x7fa248009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:09.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.257+0000 7fa25ee00640 1 --2- 172.21.15.26:0/1927315764 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x55e5d8590340 0x55e5d86273d0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:09.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.258+0000 7fa25ee00640 1 -- 172.21.15.26:0/1927315764 <== osd.3 v2:172.21.15.26:6809/1280758061 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (103) out=103b] v0'0 uv3 ondisk = 0) ==== 164+0+103 (crc 0 0 0) 0x7fa25800f030 con 0x55e5d8590340 2024-04-26T21:10:09.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.258+0000 7fa261d1ff40 1 -- 172.21.15.26:0/1927315764 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x55e5d8590340 msgr2=0x55e5d86273d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:09.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.258+0000 7fa261d1ff40 1 --2- 172.21.15.26:0/1927315764 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x55e5d8590340 0x55e5d86273d0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.258+0000 7fa261d1ff40 1 -- 172.21.15.26:0/1927315764 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa23c06b2d0 msgr2=0x7fa23c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:09.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.258+0000 7fa261d1ff40 1 --2- 172.21.15.26:0/1927315764 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa23c06b2d0 0x7fa23c06d790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55e5d8638a20 tx=0x7fa248009040 comp rx=0 tx=0).stop 2024-04-26T21:10:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.258+0000 7fa261d1ff40 1 -- 172.21.15.26:0/1927315764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5d85a8f50 msgr2=0x55e5d85a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.258+0000 7fa261d1ff40 1 --2- 172.21.15.26:0/1927315764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5d85a8f50 0x55e5d85a56a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa250025cc0 tx=0x7fa250002fb0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.258+0000 7fa25ee00640 1 -- 172.21.15.26:0/1927315764 reap_dead start 2024-04-26T21:10:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.258+0000 7fa261d1ff40 1 -- 172.21.15.26:0/1927315764 shutdown_connections 2024-04-26T21:10:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.258+0000 7fa261d1ff40 1 -- 172.21.15.26:0/1927315764 >> 172.21.15.26:0/1927315764 conn(0x55e5d858e100 msgr2=0x55e5d85aa380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.258+0000 7fa261d1ff40 1 -- 172.21.15.26:0/1927315764 shutdown_connections 2024-04-26T21:10:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.258+0000 7fa261d1ff40 1 -- 172.21.15.26:0/1927315764 wait complete. 2024-04-26T21:10:09.265 INFO:teuthology.orchestra.run.smithi026.stdout:a��alpha parent_b  2024-04-26T21:10:09.265 DEBUG:teuthology.orchestra.run.smithi026:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-26T21:10:09.361 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-26T21:10:09.361 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628283, 2024-04-26T21:10:09.361 INFO:teuthology.orchestra.run.smithi026.stdout: "ancestors": [ 2024-04-26T21:10:09.361 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:10:09.361 INFO:teuthology.orchestra.run.smithi026.stdout: "dirino": 1099511628284, 2024-04-26T21:10:09.361 INFO:teuthology.orchestra.run.smithi026.stdout: "dname": "alpha", 2024-04-26T21:10:09.361 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 9 2024-04-26T21:10:09.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:10:09.361 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:10:09.361 INFO:teuthology.orchestra.run.smithi026.stdout: "dirino": 1, 2024-04-26T21:10:09.362 INFO:teuthology.orchestra.run.smithi026.stdout: "dname": "parent_b", 2024-04-26T21:10:09.362 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 32 2024-04-26T21:10:09.362 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:10:09.362 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-26T21:10:09.362 INFO:teuthology.orchestra.run.smithi026.stdout: "pool": 6, 2024-04-26T21:10:09.362 INFO:teuthology.orchestra.run.smithi026.stdout: "old_pools": [ 2024-04-26T21:10:09.362 INFO:teuthology.orchestra.run.smithi026.stdout: 5 2024-04-26T21:10:09.362 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-26T21:10:09.362 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-26T21:10:09.362 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:09.366 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool data_new getxattr 100000001fb.00000000 parent 2024-04-26T21:10:09.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.468+0000 7f98095e4f40 1 Processor -- start 2024-04-26T21:10:09.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.468+0000 7f98095e4f40 1 -- start start 2024-04-26T21:10:09.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.469+0000 7f98095e4f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97ec0030f0 0x5567b91d9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.469+0000 7f98095e4f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567b91dfb00 0x5567b91dea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.469+0000 7f98095e4f40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5567b91def90 0x5567b9190f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.469+0000 7f98095e4f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5567b9269600 con 0x5567b91dfb00 2024-04-26T21:10:09.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.469+0000 7f98095e4f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5567b9138d20 con 0x7f97ec0030f0 2024-04-26T21:10:09.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.469+0000 7f98095e4f40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x5567b9269b80 con 0x5567b91def90 2024-04-26T21:10:09.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.470+0000 7f97fde00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567b91dfb00 0x5567b91dea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.470+0000 7f97fde00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567b91dfb00 0x5567b91dea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46786/0 (socket says 172.21.15.26:46786) 2024-04-26T21:10:09.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.470+0000 7f97fde00640 1 -- 172.21.15.26:0/2592453638 learned_addr learned my addr 172.21.15.26:0/2592453638 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:09.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.470+0000 7f97fde00640 1 -- 172.21.15.26:0/2592453638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5567b91def90 msgr2=0x5567b9190f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:09.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.470+0000 7f97fde00640 1 --2- 172.21.15.26:0/2592453638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5567b91def90 0x5567b9190f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.470+0000 7f97fde00640 1 -- 172.21.15.26:0/2592453638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97ec0030f0 msgr2=0x5567b91d9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:09.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.470+0000 7f9806800640 1 --2- 172.21.15.26:0/2592453638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97ec0030f0 0x5567b91d9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.470+0000 7f97fde00640 1 --2- 172.21.15.26:0/2592453638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97ec0030f0 0x5567b91d9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.470+0000 7f97fde00640 1 -- 172.21.15.26:0/2592453638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567b91de570 con 0x5567b91dfb00 2024-04-26T21:10:09.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.470+0000 7f9806800640 1 --2- 172.21.15.26:0/2592453638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97ec0030f0 0x5567b91d9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:09.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.470+0000 7f97fde00640 1 --2- 172.21.15.26:0/2592453638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567b91dfb00 0x5567b91dea50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f97f0025e40 tx=0x7f97f0051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382e007247877586 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:09.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.471+0000 7f9805400640 1 -- 172.21.15.26:0/2592453638 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f004f8a0 con 0x5567b91dfb00 2024-04-26T21:10:09.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.471+0000 7f9805400640 1 -- 172.21.15.26:0/2592453638 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97f004fa40 con 0x5567b91dfb00 2024-04-26T21:10:09.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.471+0000 7f9805400640 1 -- 172.21.15.26:0/2592453638 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f004fe30 con 0x5567b91dfb00 2024-04-26T21:10:09.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.471+0000 7f98095e4f40 1 -- 172.21.15.26:0/2592453638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567b91dfb00 msgr2=0x5567b91dea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:09.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.471+0000 7f98095e4f40 1 --2- 172.21.15.26:0/2592453638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567b91dfb00 0x5567b91dea50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f97f0025e40 tx=0x7f97f0051000 comp rx=0 tx=0).stop 2024-04-26T21:10:09.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.471+0000 7f98095e4f40 1 -- 172.21.15.26:0/2592453638 shutdown_connections 2024-04-26T21:10:09.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.471+0000 7f98095e4f40 1 --2- 172.21.15.26:0/2592453638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5567b91def90 0x5567b9190f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.471+0000 7f98095e4f40 1 --2- 172.21.15.26:0/2592453638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567b91dfb00 0x5567b91dea50 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.471+0000 7f98095e4f40 1 --2- 172.21.15.26:0/2592453638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97ec0030f0 0x5567b91d9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.471+0000 7f98095e4f40 1 -- 172.21.15.26:0/2592453638 >> 172.21.15.26:0/2592453638 conn(0x5567b91c1100 msgr2=0x5567b91df3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:09.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.471+0000 7f98095e4f40 1 -- 172.21.15.26:0/2592453638 shutdown_connections 2024-04-26T21:10:09.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.472+0000 7f98095e4f40 1 -- 172.21.15.26:0/2592453638 wait complete. 2024-04-26T21:10:09.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.472+0000 7f98095e4f40 1 Processor -- start 2024-04-26T21:10:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.472+0000 7f98095e4f40 1 -- start start 2024-04-26T21:10:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.473+0000 7f98095e4f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567b91def90 0x5567b91d8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.473+0000 7f98095e4f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567b91dfb00 0x5567b9256b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.473+0000 7f98095e4f40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f97ec0030f0 0x5567b9190dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.473+0000 7f98095e4f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5567b9263000 con 0x5567b91dfb00 2024-04-26T21:10:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.473+0000 7f98095e4f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5567b9262db0 con 0x5567b91def90 2024-04-26T21:10:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.473+0000 7f98095e4f40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x5567b9262900 con 0x7f97ec0030f0 2024-04-26T21:10:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.474+0000 7f97fde00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567b91dfb00 0x5567b9256b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.474+0000 7f97fde00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567b91dfb00 0x5567b9256b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46800/0 (socket says 172.21.15.26:46800) 2024-04-26T21:10:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.474+0000 7f97fde00640 1 -- 172.21.15.26:0/4043806427 learned_addr learned my addr 172.21.15.26:0/4043806427 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.474+0000 7f97fde00640 1 -- 172.21.15.26:0/4043806427 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f97ec0030f0 msgr2=0x5567b9190dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.474+0000 7f97fde00640 1 --2- 172.21.15.26:0/4043806427 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f97ec0030f0 0x5567b9190dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.474+0000 7f97fde00640 1 -- 172.21.15.26:0/4043806427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567b91def90 msgr2=0x5567b91d8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:09.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.474+0000 7f9806800640 1 --2- 172.21.15.26:0/4043806427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567b91def90 0x5567b91d8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.474+0000 7f97fde00640 1 --2- 172.21.15.26:0/4043806427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567b91def90 0x5567b91d8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.474+0000 7f97fde00640 1 -- 172.21.15.26:0/4043806427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567b925b100 con 0x5567b91dfb00 2024-04-26T21:10:09.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.474+0000 7f9806800640 1 --2- 172.21.15.26:0/4043806427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5567b91def90 0x5567b91d8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:09.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.474+0000 7f97fde00640 1 --2- 172.21.15.26:0/4043806427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567b91dfb00 0x5567b9256b00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f97f0065040 tx=0x7f97f0003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:09.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.475+0000 7f97ffe00640 1 -- 172.21.15.26:0/4043806427 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0070070 con 0x5567b91dfb00 2024-04-26T21:10:09.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.475+0000 7f97ffe00640 1 -- 172.21.15.26:0/4043806427 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97f0058400 con 0x5567b91dfb00 2024-04-26T21:10:09.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.475+0000 7f97ffe00640 1 -- 172.21.15.26:0/4043806427 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f00587f0 con 0x5567b91dfb00 2024-04-26T21:10:09.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.475+0000 7f98095e4f40 1 -- 172.21.15.26:0/4043806427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5567b91d90d0 con 0x5567b91dfb00 2024-04-26T21:10:09.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.475+0000 7f98095e4f40 1 -- 172.21.15.26:0/4043806427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567b91d80b0 con 0x5567b91dfb00 2024-04-26T21:10:09.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.477+0000 7f97ffe00640 1 -- 172.21.15.26:0/4043806427 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f97f0058990 con 0x5567b91dfb00 2024-04-26T21:10:09.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.478+0000 7f97ffe00640 1 --2- 172.21.15.26:0/4043806427 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f97e406b2d0 0x7f97e406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.478+0000 7f97ffe00640 1 -- 172.21.15.26:0/4043806427 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f97f00cf410 con 0x5567b91dfb00 2024-04-26T21:10:09.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.478+0000 7f9806800640 1 --2- 172.21.15.26:0/4043806427 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f97e406b2d0 0x7f97e406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.478+0000 7f98095e4f40 1 --2- 172.21.15.26:0/4043806427 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x5567b925a970 0x5567b9257920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.478+0000 7f98095e4f40 1 -- 172.21.15.26:0/4043806427 --> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] -- osd_op(unknown.0.0:1 6.13 6:ca8fdbba:::100000001fb.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37) -- 0x5567b9257040 con 0x5567b925a970 2024-04-26T21:10:09.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.478+0000 7f9807200640 1 --2- 172.21.15.26:0/4043806427 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x5567b925a970 0x5567b9257920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.479+0000 7f9806800640 1 --2- 172.21.15.26:0/4043806427 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f97e406b2d0 0x7f97e406d790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f97f8002250 tx=0x7f97f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:09.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.479+0000 7f9807200640 1 --2- 172.21.15.26:0/4043806427 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x5567b925a970 0x5567b9257920 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:09.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.479+0000 7f9807200640 1 -- 172.21.15.26:0/4043806427 <== osd.3 v2:172.21.15.26:6809/1280758061 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (103) out=103b] v0'0 uv2 ondisk = 0) ==== 164+0+103 (crc 0 0 0) 0x7f9800009410 con 0x5567b925a970 2024-04-26T21:10:09.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.480+0000 7f98095e4f40 1 -- 172.21.15.26:0/4043806427 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x5567b925a970 msgr2=0x5567b9257920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:09.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.480+0000 7f98095e4f40 1 --2- 172.21.15.26:0/4043806427 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x5567b925a970 0x5567b9257920 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.480+0000 7f98095e4f40 1 -- 172.21.15.26:0/4043806427 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f97e406b2d0 msgr2=0x7f97e406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:09.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.480+0000 7f98095e4f40 1 --2- 172.21.15.26:0/4043806427 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f97e406b2d0 0x7f97e406d790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f97f8002250 tx=0x7f97f8008040 comp rx=0 tx=0).stop 2024-04-26T21:10:09.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.480+0000 7f98095e4f40 1 -- 172.21.15.26:0/4043806427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567b91dfb00 msgr2=0x5567b9256b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:09.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.480+0000 7f98095e4f40 1 --2- 172.21.15.26:0/4043806427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567b91dfb00 0x5567b9256b00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f97f0065040 tx=0x7f97f0003020 comp rx=0 tx=0).stop 2024-04-26T21:10:09.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.480+0000 7f9807200640 1 -- 172.21.15.26:0/4043806427 reap_dead start 2024-04-26T21:10:09.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.480+0000 7f98095e4f40 1 -- 172.21.15.26:0/4043806427 shutdown_connections 2024-04-26T21:10:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.480+0000 7f98095e4f40 1 -- 172.21.15.26:0/4043806427 >> 172.21.15.26:0/4043806427 conn(0x5567b91c1100 msgr2=0x5567b91df3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.480+0000 7f98095e4f40 1 -- 172.21.15.26:0/4043806427 shutdown_connections 2024-04-26T21:10:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.480+0000 7f98095e4f40 1 -- 172.21.15.26:0/4043806427 wait complete. 2024-04-26T21:10:09.485 INFO:teuthology.orchestra.run.smithi026.stdout:a��alpha 2024-04-26T21:10:09.486 INFO:teuthology.orchestra.run.smithi026.stdout:parent_c' 2024-04-26T21:10:09.486 DEBUG:teuthology.orchestra.run.smithi026:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-26T21:10:09.586 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-26T21:10:09.586 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 1099511628283, 2024-04-26T21:10:09.586 INFO:teuthology.orchestra.run.smithi026.stdout: "ancestors": [ 2024-04-26T21:10:09.586 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:10:09.586 INFO:teuthology.orchestra.run.smithi026.stdout: "dirino": 1099511628285, 2024-04-26T21:10:09.586 INFO:teuthology.orchestra.run.smithi026.stdout: "dname": "alpha", 2024-04-26T21:10:09.586 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 10 2024-04-26T21:10:09.586 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-04-26T21:10:09.586 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-04-26T21:10:09.587 INFO:teuthology.orchestra.run.smithi026.stdout: "dirino": 1, 2024-04-26T21:10:09.587 INFO:teuthology.orchestra.run.smithi026.stdout: "dname": "parent_c", 2024-04-26T21:10:09.587 INFO:teuthology.orchestra.run.smithi026.stdout: "version": 39 2024-04-26T21:10:09.587 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-04-26T21:10:09.587 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-04-26T21:10:09.587 INFO:teuthology.orchestra.run.smithi026.stdout: "pool": 6, 2024-04-26T21:10:09.587 INFO:teuthology.orchestra.run.smithi026.stdout: "old_pools": [ 2024-04-26T21:10:09.587 INFO:teuthology.orchestra.run.smithi026.stdout: 5 2024-04-26T21:10:09.587 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-04-26T21:10:09.587 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-26T21:10:09.587 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:09.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:10:09.591 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.object_size -v 8388608 ./parent_c/alpha) 2024-04-26T21:10:09.647 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:10:09.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.859+0000 7f0af2c00640 1 Processor -- start 2024-04-26T21:10:09.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.859+0000 7f0af2c00640 1 -- start start 2024-04-26T21:10:09.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.861+0000 7f0af2c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0aec163930 0x7f0aec0d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.861+0000 7f0af2c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0aec0c7db0 con 0x7f0aec148aa0 2024-04-26T21:10:09.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.861+0000 7f0af2c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0aec0c8090 con 0x7f0aec0d0fa0 2024-04-26T21:10:09.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.861+0000 7f0af2c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f0aec0c8370 con 0x7f0aec163930 2024-04-26T21:10:09.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.861+0000 7f0af0e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0aec163930 0x7f0aec0d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.861+0000 7f0af1800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0aec148aa0 0x7f0aec06ee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33858/0 (socket says 172.21.15.26:33858) 2024-04-26T21:10:09.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.861+0000 7f0af1800640 1 -- 172.21.15.26:0/661818589 learned_addr learned my addr 172.21.15.26:0/661818589 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:09.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.861+0000 7f0af0e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0aec163930 0x7f0aec0d0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46804/0 (socket says 172.21.15.26:46804) 2024-04-26T21:10:09.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.862+0000 7f0af0e00640 1 -- 172.21.15.26:0/661818589 >> v1:172.21.15.86:6789/0 conn(0x7f0aec0d0fa0 legacy=0x7f0aec0d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:09.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.862+0000 7f0af0e00640 1 -- 172.21.15.26:0/661818589 >> v1:172.21.15.26:6789/0 conn(0x7f0aec148aa0 legacy=0x7f0aec06ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:09.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.862+0000 7f0af0e00640 1 -- 172.21.15.26:0/661818589 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aec06d480 con 0x7f0aec163930 2024-04-26T21:10:09.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.863+0000 7f0af0e00640 1 --2- 172.21.15.26:0/661818589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aec163930 0x7f0aec0d0a60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80097a0 tx=0x7f0ad802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba67bf7378f83e48 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:09.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.863+0000 7f0aebe00640 1 -- 172.21.15.26:0/661818589 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8037070 con 0x7f0aec163930 2024-04-26T21:10:09.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.863+0000 7f0aebe00640 1 -- 172.21.15.26:0/661818589 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ad802ae50 con 0x7f0aec163930 2024-04-26T21:10:09.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.863+0000 7f0aebe00640 1 -- 172.21.15.26:0/661818589 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8032660 con 0x7f0aec163930 2024-04-26T21:10:09.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.863+0000 7f0af2c00640 1 -- 172.21.15.26:0/661818589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aec163930 msgr2=0x7f0aec0d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:09.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.863+0000 7f0af2c00640 1 --2- 172.21.15.26:0/661818589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aec163930 0x7f0aec0d0a60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80097a0 tx=0x7f0ad802a140 comp rx=0 tx=0).stop 2024-04-26T21:10:09.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.864+0000 7f0af2c00640 1 -- 172.21.15.26:0/661818589 shutdown_connections 2024-04-26T21:10:09.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.864+0000 7f0af2c00640 1 --2- 172.21.15.26:0/661818589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aec163930 0x7f0aec0d0a60 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.864+0000 7f0af2c00640 1 -- 172.21.15.26:0/661818589 >> 172.21.15.26:0/661818589 conn(0x7f0aec075960 msgr2=0x7f0aec06c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:09.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.864+0000 7f0af2c00640 1 -- 172.21.15.26:0/661818589 shutdown_connections 2024-04-26T21:10:09.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.864+0000 7f0af2c00640 1 -- 172.21.15.26:0/661818589 wait complete. 2024-04-26T21:10:09.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.866+0000 7f0af2c00640 1 Processor -- start 2024-04-26T21:10:09.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.866+0000 7f0af2c00640 1 -- start start 2024-04-26T21:10:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.867+0000 7f0af2c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0aec0d0fa0 0x7f0aec1f7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.867+0000 7f0af2c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0aec148aa0 0x7f0aec0b4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.867+0000 7f0af2c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aec163930 0x7f0aec0b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.867+0000 7f0af2c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec0d3510 con 0x7f0aec163930 2024-04-26T21:10:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.867+0000 7f0af2c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec148e80 con 0x7f0aec0d0fa0 2024-04-26T21:10:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.867+0000 7f0af2c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0aec0d3690 con 0x7f0aec148aa0 2024-04-26T21:10:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.868+0000 7f0af2200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aec163930 0x7f0aec0b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.868+0000 7f0af2200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aec163930 0x7f0aec0b4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46808/0 (socket says 172.21.15.26:46808) 2024-04-26T21:10:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.868+0000 7f0af2200640 1 -- 172.21.15.26:0/2953991275 learned_addr learned my addr 172.21.15.26:0/2953991275 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:09.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.868+0000 7f0af0e00640 1 --2- 172.21.15.26:0/2953991275 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0aec148aa0 0x7f0aec0b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.868+0000 7f0af1800640 1 --2- 172.21.15.26:0/2953991275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0aec0d0fa0 0x7f0aec1f7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.868+0000 7f0af2200640 1 -- 172.21.15.26:0/2953991275 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0aec148aa0 msgr2=0x7f0aec0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:09.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.868+0000 7f0af2200640 1 --2- 172.21.15.26:0/2953991275 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0aec148aa0 0x7f0aec0b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.868+0000 7f0af2200640 1 -- 172.21.15.26:0/2953991275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0aec0d0fa0 msgr2=0x7f0aec1f7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:09.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.868+0000 7f0af2200640 1 --2- 172.21.15.26:0/2953991275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0aec0d0fa0 0x7f0aec1f7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:09.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.868+0000 7f0af2200640 1 -- 172.21.15.26:0/2953991275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aec1604c0 con 0x7f0aec163930 2024-04-26T21:10:09.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.868+0000 7f0af2200640 1 --2- 172.21.15.26:0/2953991275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aec163930 0x7f0aec0b4b50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400ba10 tx=0x7f0ae400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:09.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.869+0000 7f0ae9600640 1 -- 172.21.15.26:0/2953991275 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4019070 con 0x7f0aec163930 2024-04-26T21:10:09.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.869+0000 7f0ae9600640 1 -- 172.21.15.26:0/2953991275 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ae400cb40 con 0x7f0aec163930 2024-04-26T21:10:09.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.869+0000 7f0ae9600640 1 -- 172.21.15.26:0/2953991275 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4014440 con 0x7f0aec163930 2024-04-26T21:10:09.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.869+0000 7f0af2c00640 1 -- 172.21.15.26:0/2953991275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aec1607a0 con 0x7f0aec163930 2024-04-26T21:10:09.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.869+0000 7f0af2c00640 1 -- 172.21.15.26:0/2953991275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aec160d60 con 0x7f0aec163930 2024-04-26T21:10:09.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.870+0000 7f0af2c00640 1 -- 172.21.15.26:0/2953991275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aec0693d0 con 0x7f0aec163930 2024-04-26T21:10:09.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.871+0000 7f0ae9600640 1 -- 172.21.15.26:0/2953991275 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ae401c070 con 0x7f0aec163930 2024-04-26T21:10:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.872+0000 7f0ae9600640 1 --2- 172.21.15.26:0/2953991275 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0acc06c8f0 0x7f0acc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.872+0000 7f0ae9600640 1 -- 172.21.15.26:0/2953991275 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f0ae408b940 con 0x7f0aec163930 2024-04-26T21:10:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.872+0000 7f0af1800640 1 --2- 172.21.15.26:0/2953991275 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0acc06c8f0 0x7f0acc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.873+0000 7f0af1800640 1 --2- 172.21.15.26:0/2953991275 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0acc06c8f0 0x7f0acc06edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0aec06c870 tx=0x7f0ae0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:09.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:09.877+0000 7f0ae9600640 1 -- 172.21.15.26:0/2953991275 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0ae4059b40 con 0x7f0aec163930 2024-04-26T21:10:10.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.121+0000 7f0af2c00640 1 -- 172.21.15.26:0/2953991275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0aec1455c0 con 0x7f0aec163930 2024-04-26T21:10:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.122+0000 7f0ae9600640 1 -- 172.21.15.26:0/2953991275 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5180 (secure 0 0 0) 0x7f0ae4059440 con 0x7f0aec163930 2024-04-26T21:10:10.124 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:10.124 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2349160700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2349160700},{"type":"v1","addr":"172.21.15.26:6837","nonce":2349160700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4598,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2824191762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2824191762},{"type":"v1","addr":"172.21.15.86:6834","nonce":2824191762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/263746310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":263746310},{"type":"v1","addr":"172.21.15.86:6835","nonce":263746310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:09:48.622025+0000","modified":"2024-04-26T21:10:05.296141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5058":{"gid":5058,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6836/2528180854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2528180854},{"type":"v1","addr":"172.21.15.26:6836","nonce":2528180854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5058,"qdb_cluster":[5058]},"id":2}]} 2024-04-26T21:10:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 18 2024-04-26T21:10:10.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.125+0000 7f0af2c00640 1 -- 172.21.15.26:0/2953991275 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0acc06c8f0 msgr2=0x7f0acc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:10.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.125+0000 7f0af2c00640 1 --2- 172.21.15.26:0/2953991275 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0acc06c8f0 0x7f0acc06edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0aec06c870 tx=0x7f0ae0009040 comp rx=0 tx=0).stop 2024-04-26T21:10:10.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.125+0000 7f0af2c00640 1 -- 172.21.15.26:0/2953991275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aec163930 msgr2=0x7f0aec0b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:10.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.125+0000 7f0af2c00640 1 --2- 172.21.15.26:0/2953991275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aec163930 0x7f0aec0b4b50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400ba10 tx=0x7f0ae400bed0 comp rx=0 tx=0).stop 2024-04-26T21:10:10.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.126+0000 7f0af2c00640 1 -- 172.21.15.26:0/2953991275 shutdown_connections 2024-04-26T21:10:10.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.126+0000 7f0af2c00640 1 --2- 172.21.15.26:0/2953991275 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0acc06c8f0 0x7f0acc06edb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:10.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.126+0000 7f0af2c00640 1 --2- 172.21.15.26:0/2953991275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aec163930 0x7f0aec0b4b50 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:10.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.126+0000 7f0af2c00640 1 --2- 172.21.15.26:0/2953991275 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0aec148aa0 0x7f0aec0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:10.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.126+0000 7f0af2c00640 1 --2- 172.21.15.26:0/2953991275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0aec0d0fa0 0x7f0aec1f7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:10.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.126+0000 7f0af2c00640 1 -- 172.21.15.26:0/2953991275 >> 172.21.15.26:0/2953991275 conn(0x7f0aec075960 msgr2=0x7f0aec070540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:10.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.126+0000 7f0af2c00640 1 -- 172.21.15.26:0/2953991275 shutdown_connections 2024-04-26T21:10:10.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.126+0000 7f0af2c00640 1 -- 172.21.15.26:0/2953991275 wait complete. 2024-04-26T21:10:10.148 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-26T21:10:10.395 INFO:teuthology.orchestra.run.smithi026.stdout:{"message":"","return_code":0} 2024-04-26T21:10:10.396 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-26T21:10:10.396 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool data_new getxattr 100000001fb.00000000 layout 2024-04-26T21:10:10.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.504+0000 7f687f3d6f40 1 Processor -- start 2024-04-26T21:10:10.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.504+0000 7f687f3d6f40 1 -- start start 2024-04-26T21:10:10.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.505+0000 7f687f3d6f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f481906f50 0x55f481904890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:10.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.505+0000 7f687f3d6f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f481994050 0x55f4818ba310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:10.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.505+0000 7f687f3d6f40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f481996eb0 0x55f4818eae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:10.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.505+0000 7f687f3d6f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f481994600 con 0x55f481994050 2024-04-26T21:10:10.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.505+0000 7f687f3d6f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f481909e70 con 0x55f481906f50 2024-04-26T21:10:10.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.505+0000 7f687f3d6f40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55f481994b80 con 0x55f481996eb0 2024-04-26T21:10:10.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.506+0000 7f6873c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f481994050 0x55f4818ba310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:10.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.506+0000 7f6873c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f481994050 0x55f4818ba310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46834/0 (socket says 172.21.15.26:46834) 2024-04-26T21:10:10.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.506+0000 7f6873c00640 1 -- 172.21.15.26:0/2638543216 learned_addr learned my addr 172.21.15.26:0/2638543216 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:10.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.506+0000 7f687d000640 1 --2- 172.21.15.26:0/2638543216 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f481996eb0 0x55f4818eae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:10.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.506+0000 7f687c600640 1 --2- 172.21.15.26:0/2638543216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f481906f50 0x55f481904890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:10.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.506+0000 7f6873c00640 1 -- 172.21.15.26:0/2638543216 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f481996eb0 msgr2=0x55f4818eae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:10.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.506+0000 7f6873c00640 1 --2- 172.21.15.26:0/2638543216 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f481996eb0 0x55f4818eae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:10.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.506+0000 7f6873c00640 1 -- 172.21.15.26:0/2638543216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f481906f50 msgr2=0x55f481904890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:10.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.506+0000 7f6873c00640 1 --2- 172.21.15.26:0/2638543216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f481906f50 0x55f481904890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:10.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.506+0000 7f6873c00640 1 -- 172.21.15.26:0/2638543216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4817e6b30 con 0x55f481994050 2024-04-26T21:10:10.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.506+0000 7f687d000640 1 --2- 172.21.15.26:0/2638543216 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f481996eb0 0x55f4818eae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:10.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.506+0000 7f687c600640 1 --2- 172.21.15.26:0/2638543216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f481906f50 0x55f481904890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:10.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.507+0000 7f6873c00640 1 --2- 172.21.15.26:0/2638543216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f481994050 0x55f4818ba310 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f686400a770 tx=0x7f686400a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2105f71b9bcdca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:10.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.507+0000 7f687b200640 1 -- 172.21.15.26:0/2638543216 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864019070 con 0x55f481994050 2024-04-26T21:10:10.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.507+0000 7f687b200640 1 -- 172.21.15.26:0/2638543216 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6864007600 con 0x55f481994050 2024-04-26T21:10:10.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.507+0000 7f687b200640 1 -- 172.21.15.26:0/2638543216 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68640079d0 con 0x55f481994050 2024-04-26T21:10:10.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.508+0000 7f687f3d6f40 1 -- 172.21.15.26:0/2638543216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f481994050 msgr2=0x55f4818ba310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:10.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.508+0000 7f687f3d6f40 1 --2- 172.21.15.26:0/2638543216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f481994050 0x55f4818ba310 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f686400a770 tx=0x7f686400a860 comp rx=0 tx=0).stop 2024-04-26T21:10:10.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.508+0000 7f687f3d6f40 1 -- 172.21.15.26:0/2638543216 shutdown_connections 2024-04-26T21:10:10.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.508+0000 7f687f3d6f40 1 --2- 172.21.15.26:0/2638543216 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f481996eb0 0x55f4818eae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:10.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.508+0000 7f687f3d6f40 1 --2- 172.21.15.26:0/2638543216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f481994050 0x55f4818ba310 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:10.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.508+0000 7f687f3d6f40 1 --2- 172.21.15.26:0/2638543216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f481906f50 0x55f481904890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:10.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.508+0000 7f687f3d6f40 1 -- 172.21.15.26:0/2638543216 >> 172.21.15.26:0/2638543216 conn(0x55f4818ec100 msgr2=0x55f4819965c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:10.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.508+0000 7f687f3d6f40 1 -- 172.21.15.26:0/2638543216 shutdown_connections 2024-04-26T21:10:10.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.508+0000 7f687f3d6f40 1 -- 172.21.15.26:0/2638543216 wait complete. 2024-04-26T21:10:10.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.509+0000 7f687f3d6f40 1 Processor -- start 2024-04-26T21:10:10.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.509+0000 7f687f3d6f40 1 -- start start 2024-04-26T21:10:10.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.510+0000 7f687f3d6f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f481906f50 0x55f481985d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:10.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.510+0000 7f687f3d6f40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f481994050 0x55f481a2a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:10.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.510+0000 7f687f3d6f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f481996eb0 0x55f4818ba1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:10.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.510+0000 7f687f3d6f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f4819866a0 con 0x55f481906f50 2024-04-26T21:10:10.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.510+0000 7f687f3d6f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f4818bb850 con 0x55f481996eb0 2024-04-26T21:10:10.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.510+0000 7f687f3d6f40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x55f4818bad90 con 0x55f481994050 2024-04-26T21:10:10.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.510+0000 7f687c600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f481906f50 0x55f481985d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:10.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.510+0000 7f687c600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f481906f50 0x55f481985d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46840/0 (socket says 172.21.15.26:46840) 2024-04-26T21:10:10.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.510+0000 7f687c600640 1 -- 172.21.15.26:0/730119759 learned_addr learned my addr 172.21.15.26:0/730119759 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:10.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.510+0000 7f6873c00640 1 --2- 172.21.15.26:0/730119759 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f481994050 0x55f481a2a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:10.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.510+0000 7f687c600640 1 -- 172.21.15.26:0/730119759 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f481994050 msgr2=0x55f481a2a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:10.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.510+0000 7f687d000640 1 --2- 172.21.15.26:0/730119759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f481996eb0 0x55f4818ba1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:10.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.510+0000 7f687c600640 1 --2- 172.21.15.26:0/730119759 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f481994050 0x55f481a2a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:10.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.510+0000 7f687c600640 1 -- 172.21.15.26:0/730119759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f481996eb0 msgr2=0x55f4818ba1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:10.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.511+0000 7f687c600640 1 --2- 172.21.15.26:0/730119759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f481996eb0 0x55f4818ba1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:10.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.511+0000 7f687c600640 1 -- 172.21.15.26:0/730119759 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4818ef5e0 con 0x55f481906f50 2024-04-26T21:10:10.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.511+0000 7f6873c00640 1 --2- 172.21.15.26:0/730119759 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f481994050 0x55f481a2a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:10.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.511+0000 7f687d000640 1 --2- 172.21.15.26:0/730119759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f481996eb0 0x55f4818ba1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:10.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.511+0000 7f687c600640 1 --2- 172.21.15.26:0/730119759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f481906f50 0x55f481985d60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f686c008420 tx=0x7f686c0088e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:10.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.511+0000 7f6879e00640 1 -- 172.21.15.26:0/730119759 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c00e040 con 0x55f481906f50 2024-04-26T21:10:10.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.511+0000 7f6879e00640 1 -- 172.21.15.26:0/730119759 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f686c017070 con 0x55f481906f50 2024-04-26T21:10:10.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.511+0000 7f687f3d6f40 1 -- 172.21.15.26:0/730119759 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f4818efa40 con 0x55f481906f50 2024-04-26T21:10:10.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.511+0000 7f6879e00640 1 -- 172.21.15.26:0/730119759 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c010870 con 0x55f481906f50 2024-04-26T21:10:10.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.511+0000 7f687f3d6f40 1 -- 172.21.15.26:0/730119759 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4818ef300 con 0x55f481906f50 2024-04-26T21:10:10.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.514+0000 7f6879e00640 1 -- 172.21.15.26:0/730119759 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f686c010a10 con 0x55f481906f50 2024-04-26T21:10:10.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.514+0000 7f6879e00640 1 --2- 172.21.15.26:0/730119759 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f685806b300 0x7f685806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:10.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.515+0000 7f6879e00640 1 -- 172.21.15.26:0/730119759 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f686c08a770 con 0x55f481906f50 2024-04-26T21:10:10.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.515+0000 7f6873c00640 1 --2- 172.21.15.26:0/730119759 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f685806b300 0x7f685806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:10.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.515+0000 7f687f3d6f40 1 --2- 172.21.15.26:0/730119759 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x55f481996ab0 0x55f481902630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:10.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.515+0000 7f687f3d6f40 1 -- 172.21.15.26:0/730119759 --> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] -- osd_op(unknown.0.0:1 6.13 6:ca8fdbba:::100000001fb.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37) -- 0x55f481907c00 con 0x55f481996ab0 2024-04-26T21:10:10.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.515+0000 7f687d000640 1 --2- 172.21.15.26:0/730119759 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x55f481996ab0 0x55f481902630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:10.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.516+0000 7f6873c00640 1 --2- 172.21.15.26:0/730119759 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f685806b300 0x7f685806d7c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55f481909480 tx=0x7f686400c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:10.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.516+0000 7f687d000640 1 --2- 172.21.15.26:0/730119759 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x55f481996ab0 0x55f481902630 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:10.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.516+0000 7f687d000640 1 -- 172.21.15.26:0/730119759 <== osd.3 v2:172.21.15.26:6809/1280758061 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) ==== 164+0+30 (crc 0 0 0) 0x7f6874054970 con 0x55f481996ab0 2024-04-26T21:10:10.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.517+0000 7f687f3d6f40 1 -- 172.21.15.26:0/730119759 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x55f481996ab0 msgr2=0x55f481902630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:10.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.517+0000 7f687f3d6f40 1 --2- 172.21.15.26:0/730119759 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x55f481996ab0 0x55f481902630 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:10.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.517+0000 7f687f3d6f40 1 -- 172.21.15.26:0/730119759 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f685806b300 msgr2=0x7f685806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:10.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.517+0000 7f687f3d6f40 1 --2- 172.21.15.26:0/730119759 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f685806b300 0x7f685806d7c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55f481909480 tx=0x7f686400c000 comp rx=0 tx=0).stop 2024-04-26T21:10:10.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.517+0000 7f687f3d6f40 1 -- 172.21.15.26:0/730119759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f481906f50 msgr2=0x55f481985d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:10.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.517+0000 7f687f3d6f40 1 --2- 172.21.15.26:0/730119759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f481906f50 0x55f481985d60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f686c008420 tx=0x7f686c0088e0 comp rx=0 tx=0).stop 2024-04-26T21:10:10.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.517+0000 7f687d000640 1 -- 172.21.15.26:0/730119759 reap_dead start 2024-04-26T21:10:10.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.517+0000 7f687f3d6f40 1 -- 172.21.15.26:0/730119759 shutdown_connections 2024-04-26T21:10:10.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.517+0000 7f687f3d6f40 1 -- 172.21.15.26:0/730119759 >> 172.21.15.26:0/730119759 conn(0x55f4818ec100 msgr2=0x55f4818aa5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:10.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.517+0000 7f687f3d6f40 1 -- 172.21.15.26:0/730119759 shutdown_connections 2024-04-26T21:10:10.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.517+0000 7f687f3d6f40 1 -- 172.21.15.26:0/730119759 wait complete. 2024-04-26T21:10:10.523 INFO:teuthology.orchestra.run.smithi026.stdout:@� 2024-04-26T21:10:10.524 DEBUG:teuthology.orchestra.run.smithi026:> ceph-dencoder type file_layout_t import - decode dump_json 2024-04-26T21:10:10.631 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-26T21:10:10.631 INFO:teuthology.orchestra.run.smithi026.stdout: "stripe_unit": 4194304, 2024-04-26T21:10:10.631 INFO:teuthology.orchestra.run.smithi026.stdout: "stripe_count": 1, 2024-04-26T21:10:10.631 INFO:teuthology.orchestra.run.smithi026.stdout: "object_size": 8388608, 2024-04-26T21:10:10.631 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_id": 6, 2024-04-26T21:10:10.631 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_ns": "" 2024-04-26T21:10:10.631 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-26T21:10:10.631 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:10.636 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001fb.00000000 layout 2024-04-26T21:10:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.751+0000 7efd3b292f40 1 Processor -- start 2024-04-26T21:10:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.751+0000 7efd3b292f40 1 -- start start 2024-04-26T21:10:10.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.752+0000 7efd3b292f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f8383bf50 0x559f838200f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:10.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.752+0000 7efd3b292f40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559f838c9050 0x559f838ca420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:10.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.752+0000 7efd3b292f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f838cbeb0 0x559f8381eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:10.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.752+0000 7efd3b292f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559f8383ee70 con 0x559f8383bf50 2024-04-26T21:10:10.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.752+0000 7efd3b292f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559f83798d20 con 0x559f838cbeb0 2024-04-26T21:10:10.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.752+0000 7efd3b292f40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x559f838c9600 con 0x559f838c9050 2024-04-26T21:10:10.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.753+0000 7efd38400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f8383bf50 0x559f838200f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:10.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.753+0000 7efd38400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f8383bf50 0x559f838200f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46852/0 (socket says 172.21.15.26:46852) 2024-04-26T21:10:10.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.753+0000 7efd38400640 1 -- 172.21.15.26:0/2172163804 learned_addr learned my addr 172.21.15.26:0/2172163804 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:10.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.753+0000 7efd37a00640 1 --2- 172.21.15.26:0/2172163804 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559f838c9050 0x559f838ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:10.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.753+0000 7efd38e00640 1 --2- 172.21.15.26:0/2172163804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f838cbeb0 0x559f8381eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:10.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.753+0000 7efd38400640 1 -- 172.21.15.26:0/2172163804 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559f838c9050 msgr2=0x559f838ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:10.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.753+0000 7efd38400640 1 --2- 172.21.15.26:0/2172163804 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559f838c9050 0x559f838ca420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:10.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.753+0000 7efd38400640 1 -- 172.21.15.26:0/2172163804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f838cbeb0 msgr2=0x559f8381eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:10.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.753+0000 7efd38400640 1 --2- 172.21.15.26:0/2172163804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f838cbeb0 0x559f8381eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:10.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.753+0000 7efd38400640 1 -- 172.21.15.26:0/2172163804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f838bb6a0 con 0x559f8383bf50 2024-04-26T21:10:10.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.753+0000 7efd37a00640 1 --2- 172.21.15.26:0/2172163804 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559f838c9050 0x559f838ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:10.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.753+0000 7efd38e00640 1 --2- 172.21.15.26:0/2172163804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f838cbeb0 0x559f8381eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:10.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.753+0000 7efd38400640 1 --2- 172.21.15.26:0/2172163804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f8383bf50 0x559f838200f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7efd2800a770 tx=0x7efd2800a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4b28852d78d673 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:10.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.754+0000 7efd36600640 1 -- 172.21.15.26:0/2172163804 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28019070 con 0x559f8383bf50 2024-04-26T21:10:10.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.754+0000 7efd36600640 1 -- 172.21.15.26:0/2172163804 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd28007600 con 0x559f8383bf50 2024-04-26T21:10:10.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.754+0000 7efd36600640 1 -- 172.21.15.26:0/2172163804 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd280079d0 con 0x559f8383bf50 2024-04-26T21:10:10.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.754+0000 7efd3b292f40 1 -- 172.21.15.26:0/2172163804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f8383bf50 msgr2=0x559f838200f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:10.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.754+0000 7efd3b292f40 1 --2- 172.21.15.26:0/2172163804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f8383bf50 0x559f838200f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7efd2800a770 tx=0x7efd2800a860 comp rx=0 tx=0).stop 2024-04-26T21:10:10.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.755+0000 7efd3b292f40 1 -- 172.21.15.26:0/2172163804 shutdown_connections 2024-04-26T21:10:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.755+0000 7efd3b292f40 1 --2- 172.21.15.26:0/2172163804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f838cbeb0 0x559f8381eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.755+0000 7efd3b292f40 1 --2- 172.21.15.26:0/2172163804 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559f838c9050 0x559f838ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.755+0000 7efd3b292f40 1 --2- 172.21.15.26:0/2172163804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f8383bf50 0x559f838200f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.755+0000 7efd3b292f40 1 -- 172.21.15.26:0/2172163804 >> 172.21.15.26:0/2172163804 conn(0x559f83821100 msgr2=0x559f838cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.755+0000 7efd3b292f40 1 -- 172.21.15.26:0/2172163804 shutdown_connections 2024-04-26T21:10:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.755+0000 7efd3b292f40 1 -- 172.21.15.26:0/2172163804 wait complete. 2024-04-26T21:10:10.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.755+0000 7efd3b292f40 1 Processor -- start 2024-04-26T21:10:10.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.756+0000 7efd3b292f40 1 -- start start 2024-04-26T21:10:10.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.756+0000 7efd3b292f40 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f8383bf50 0x559f83838d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:10.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.756+0000 7efd3b292f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f838c9050 0x559f8383d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:10.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.757+0000 7efd3b292f40 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559f838cbeb0 0x559f8383d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:10.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.757+0000 7efd3b292f40 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559f837f1710 con 0x559f8383bf50 2024-04-26T21:10:10.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.757+0000 7efd3b292f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559f837f1430 con 0x559f838c9050 2024-04-26T21:10:10.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.757+0000 7efd3b292f40 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x559f838bb960 con 0x559f838cbeb0 2024-04-26T21:10:10.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.757+0000 7efd38400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f8383bf50 0x559f83838d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:10.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.757+0000 7efd38400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f8383bf50 0x559f83838d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46856/0 (socket says 172.21.15.26:46856) 2024-04-26T21:10:10.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.757+0000 7efd38400640 1 -- 172.21.15.26:0/1535435871 learned_addr learned my addr 172.21.15.26:0/1535435871 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:10.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.757+0000 7efd37a00640 1 --2- 172.21.15.26:0/1535435871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f838c9050 0x559f8383d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:10.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.757+0000 7efd38e00640 1 --2- 172.21.15.26:0/1535435871 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559f838cbeb0 0x559f8383d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:10.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.757+0000 7efd38400640 1 -- 172.21.15.26:0/1535435871 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559f838cbeb0 msgr2=0x559f8383d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:10.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.757+0000 7efd38400640 1 --2- 172.21.15.26:0/1535435871 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559f838cbeb0 0x559f8383d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:10.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.757+0000 7efd38400640 1 -- 172.21.15.26:0/1535435871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f838c9050 msgr2=0x559f8383d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:10.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.757+0000 7efd38400640 1 --2- 172.21.15.26:0/1535435871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f838c9050 0x559f8383d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:10.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.757+0000 7efd38400640 1 -- 172.21.15.26:0/1535435871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f83957d60 con 0x559f8383bf50 2024-04-26T21:10:10.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.757+0000 7efd37a00640 1 --2- 172.21.15.26:0/1535435871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f838c9050 0x559f8383d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:10.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.758+0000 7efd38400640 1 --2- 172.21.15.26:0/1535435871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f8383bf50 0x559f83838d60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7efd2800ad60 tx=0x7efd28003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:10.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.758+0000 7efd35200640 1 -- 172.21.15.26:0/1535435871 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28019040 con 0x559f8383bf50 2024-04-26T21:10:10.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.758+0000 7efd35200640 1 -- 172.21.15.26:0/1535435871 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd28022070 con 0x559f8383bf50 2024-04-26T21:10:10.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.758+0000 7efd35200640 1 -- 172.21.15.26:0/1535435871 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd280146b0 con 0x559f8383bf50 2024-04-26T21:10:10.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.758+0000 7efd3b292f40 1 -- 172.21.15.26:0/1535435871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559f838371d0 con 0x559f8383bf50 2024-04-26T21:10:10.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.758+0000 7efd3b292f40 1 -- 172.21.15.26:0/1535435871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f83958040 con 0x559f8383bf50 2024-04-26T21:10:10.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.760+0000 7efd35200640 1 -- 172.21.15.26:0/1535435871 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd28014850 con 0x559f8383bf50 2024-04-26T21:10:10.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.761+0000 7efd35200640 1 --2- 172.21.15.26:0/1535435871 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7efd1806b2d0 0x7efd1806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:10.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.761+0000 7efd35200640 1 -- 172.21.15.26:0/1535435871 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7efd2808b7c0 con 0x559f8383bf50 2024-04-26T21:10:10.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.762+0000 7efd37a00640 1 --2- 172.21.15.26:0/1535435871 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7efd1806b2d0 0x7efd1806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.762+0000 7efd3b292f40 1 --2- 172.21.15.26:0/1535435871 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x559f83837cb0 0x559f8395fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.762+0000 7efd3b292f40 1 -- 172.21.15.26:0/1535435871 --> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] -- osd_op(unknown.0.0:1 5.13 5:ca8fdbba:::100000001fb.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37) -- 0x559f8395ffa0 con 0x559f83837cb0 2024-04-26T21:10:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.762+0000 7efd38e00640 1 --2- 172.21.15.26:0/1535435871 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x559f83837cb0 0x559f8395fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.763+0000 7efd37a00640 1 --2- 172.21.15.26:0/1535435871 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7efd1806b2d0 0x7efd1806d790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efd24057000 tx=0x7efd24052000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.763+0000 7efd38e00640 1 --2- 172.21.15.26:0/1535435871 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x559f83837cb0 0x559f8395fa60 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.763+0000 7efd38e00640 1 -- 172.21.15.26:0/1535435871 <== osd.3 v2:172.21.15.26:6809/1280758061 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) ==== 164+0+30 (crc 0 0 0) 0x7efd3000c580 con 0x559f83837cb0 2024-04-26T21:10:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.763+0000 7efd3b292f40 1 -- 172.21.15.26:0/1535435871 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x559f83837cb0 msgr2=0x559f8395fa60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.763+0000 7efd3b292f40 1 --2- 172.21.15.26:0/1535435871 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x559f83837cb0 0x559f8395fa60 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.763+0000 7efd3b292f40 1 -- 172.21.15.26:0/1535435871 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7efd1806b2d0 msgr2=0x7efd1806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.763+0000 7efd3b292f40 1 --2- 172.21.15.26:0/1535435871 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7efd1806b2d0 0x7efd1806d790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efd24057000 tx=0x7efd24052000 comp rx=0 tx=0).stop 2024-04-26T21:10:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.763+0000 7efd3b292f40 1 -- 172.21.15.26:0/1535435871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f8383bf50 msgr2=0x559f83838d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.763+0000 7efd3b292f40 1 --2- 172.21.15.26:0/1535435871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559f8383bf50 0x559f83838d60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7efd2800ad60 tx=0x7efd28003020 comp rx=0 tx=0).stop 2024-04-26T21:10:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.763+0000 7efd38e00640 1 -- 172.21.15.26:0/1535435871 reap_dead start 2024-04-26T21:10:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.764+0000 7efd3b292f40 1 -- 172.21.15.26:0/1535435871 shutdown_connections 2024-04-26T21:10:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.764+0000 7efd3b292f40 1 -- 172.21.15.26:0/1535435871 >> 172.21.15.26:0/1535435871 conn(0x559f83821100 msgr2=0x559f838200f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.764+0000 7efd3b292f40 1 -- 172.21.15.26:0/1535435871 shutdown_connections 2024-04-26T21:10:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:10.764+0000 7efd3b292f40 1 -- 172.21.15.26:0/1535435871 wait complete. 2024-04-26T21:10:10.770 INFO:teuthology.orchestra.run.smithi026.stdout:@@ 2024-04-26T21:10:10.770 DEBUG:teuthology.orchestra.run.smithi026:> ceph-dencoder type file_layout_t import - decode dump_json 2024-04-26T21:10:10.880 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-26T21:10:10.880 INFO:teuthology.orchestra.run.smithi026.stdout: "stripe_unit": 4194304, 2024-04-26T21:10:10.880 INFO:teuthology.orchestra.run.smithi026.stdout: "stripe_count": 1, 2024-04-26T21:10:10.880 INFO:teuthology.orchestra.run.smithi026.stdout: "object_size": 4194304, 2024-04-26T21:10:10.880 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_id": 5, 2024-04-26T21:10:10.880 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_ns": "" 2024-04-26T21:10:10.880 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-26T21:10:10.880 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:10.886 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:10:10.886 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:10:11.004 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:10:11.117 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:10:11.118 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:10:11.118 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:10:11.118 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:10:11.179 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:10:11.179 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:10:11.237 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:10:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.451+0000 7fd547200640 1 Processor -- start 2024-04-26T21:10:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.452+0000 7fd547200640 1 -- start start 2024-04-26T21:10:11.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.453+0000 7fd547200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd54806e700 0x7fd54806ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:11.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.453+0000 7fd547200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd548072500 con 0x7fd5481677a0 2024-04-26T21:10:11.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.453+0000 7fd547200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5480727e0 con 0x7fd54815dcb0 2024-04-26T21:10:11.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.453+0000 7fd547200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fd54815e090 con 0x7fd54806e700 2024-04-26T21:10:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.454+0000 7fd546800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd5481677a0 0x7fd5480d1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33862/0 (socket says 172.21.15.26:33862) 2024-04-26T21:10:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.454+0000 7fd546800640 1 -- 172.21.15.26:0/3152129301 learned_addr learned my addr 172.21.15.26:0/3152129301 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.454+0000 7fd545400640 1 --2- 172.21.15.26:0/3152129301 >> v2:172.21.15.26:3300/0 conn(0x7fd54806e700 0x7fd54806ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.454+0000 7fd544a00640 1 -- 172.21.15.26:0/3152129301 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 49566139 0 0) 0x7fd548072500 con 0x7fd5481677a0 2024-04-26T21:10:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.454+0000 7fd544a00640 1 -- 172.21.15.26:0/3152129301 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd52c003520 con 0x7fd5481677a0 2024-04-26T21:10:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.454+0000 7fd545400640 1 -- 172.21.15.26:0/3152129301 >> v1:172.21.15.86:6789/0 conn(0x7fd54815dcb0 legacy=0x7fd548164590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.455+0000 7fd545400640 1 -- 172.21.15.26:0/3152129301 >> v1:172.21.15.26:6789/0 conn(0x7fd5481677a0 legacy=0x7fd5480d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.455+0000 7fd545400640 1 -- 172.21.15.26:0/3152129301 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd548072ac0 con 0x7fd54806e700 2024-04-26T21:10:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.456+0000 7fd545400640 1 --2- 172.21.15.26:0/3152129301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54806e700 0x7fd54806ebb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd538009d90 tx=0x7fd53802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3714e4829cf99105 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.456+0000 7fd544a00640 1 -- 172.21.15.26:0/3152129301 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd538031600 con 0x7fd54806e700 2024-04-26T21:10:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.456+0000 7fd544a00640 1 -- 172.21.15.26:0/3152129301 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5380317a0 con 0x7fd54806e700 2024-04-26T21:10:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.456+0000 7fd544a00640 1 -- 172.21.15.26:0/3152129301 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd538031b70 con 0x7fd54806e700 2024-04-26T21:10:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.457+0000 7fd547200640 1 -- 172.21.15.26:0/3152129301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54806e700 msgr2=0x7fd54806ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.457+0000 7fd547200640 1 --2- 172.21.15.26:0/3152129301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54806e700 0x7fd54806ebb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd538009d90 tx=0x7fd53802a5d0 comp rx=0 tx=0).stop 2024-04-26T21:10:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.457+0000 7fd547200640 1 -- 172.21.15.26:0/3152129301 shutdown_connections 2024-04-26T21:10:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.457+0000 7fd547200640 1 --2- 172.21.15.26:0/3152129301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54806e700 0x7fd54806ebb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.457+0000 7fd547200640 1 -- 172.21.15.26:0/3152129301 >> 172.21.15.26:0/3152129301 conn(0x7fd54806c690 msgr2=0x7fd54806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.457+0000 7fd547200640 1 -- 172.21.15.26:0/3152129301 shutdown_connections 2024-04-26T21:10:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.457+0000 7fd547200640 1 -- 172.21.15.26:0/3152129301 wait complete. 2024-04-26T21:10:11.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.459+0000 7fd547200640 1 Processor -- start 2024-04-26T21:10:11.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.459+0000 7fd547200640 1 -- start start 2024-04-26T21:10:11.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.461+0000 7fd547200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54806e700 0x7fd5481606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.461+0000 7fd547200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd54815dcb0 0x7fd54815ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.461+0000 7fd547200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5481677a0 0x7fd54815f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.461+0000 7fd547200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd54806e3c0 con 0x7fd54806e700 2024-04-26T21:10:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.461+0000 7fd547200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd548072da0 con 0x7fd5481677a0 2024-04-26T21:10:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.461+0000 7fd547200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fd5480721a0 con 0x7fd54815dcb0 2024-04-26T21:10:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.461+0000 7fd545e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54806e700 0x7fd5481606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.461+0000 7fd545e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54806e700 0x7fd5481606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46876/0 (socket says 172.21.15.26:46876) 2024-04-26T21:10:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.461+0000 7fd545e00640 1 -- 172.21.15.26:0/3851023635 learned_addr learned my addr 172.21.15.26:0/3851023635 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.461+0000 7fd545400640 1 --2- 172.21.15.26:0/3851023635 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd54815dcb0 0x7fd54815ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.461+0000 7fd546800640 1 --2- 172.21.15.26:0/3851023635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5481677a0 0x7fd54815f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:11.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.462+0000 7fd545e00640 1 -- 172.21.15.26:0/3851023635 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd54815dcb0 msgr2=0x7fd54815ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:11.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.462+0000 7fd545e00640 1 --2- 172.21.15.26:0/3851023635 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd54815dcb0 0x7fd54815ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.462+0000 7fd545e00640 1 -- 172.21.15.26:0/3851023635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5481677a0 msgr2=0x7fd54815f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.462+0000 7fd545e00640 1 --2- 172.21.15.26:0/3851023635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5481677a0 0x7fd54815f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.462+0000 7fd545e00640 1 -- 172.21.15.26:0/3851023635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd548151420 con 0x7fd54806e700 2024-04-26T21:10:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.462+0000 7fd545e00640 1 --2- 172.21.15.26:0/3851023635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54806e700 0x7fd5481606a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd53000c480 tx=0x7fd53000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.462+0000 7fd53e000640 1 -- 172.21.15.26:0/3851023635 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5300052d0 con 0x7fd54806e700 2024-04-26T21:10:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.462+0000 7fd53e000640 1 -- 172.21.15.26:0/3851023635 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd530005470 con 0x7fd54806e700 2024-04-26T21:10:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.462+0000 7fd547200640 1 -- 172.21.15.26:0/3851023635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd54815fd60 con 0x7fd54806e700 2024-04-26T21:10:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.462+0000 7fd53e000640 1 -- 172.21.15.26:0/3851023635 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530012870 con 0x7fd54806e700 2024-04-26T21:10:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.463+0000 7fd547200640 1 -- 172.21.15.26:0/3851023635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd54815fa80 con 0x7fd54806e700 2024-04-26T21:10:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.464+0000 7fd547200640 1 -- 172.21.15.26:0/3851023635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd54806a700 con 0x7fd54806e700 2024-04-26T21:10:11.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.465+0000 7fd53e000640 1 -- 172.21.15.26:0/3851023635 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd530012a10 con 0x7fd54806e700 2024-04-26T21:10:11.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.466+0000 7fd53e000640 1 --2- 172.21.15.26:0/3851023635 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd52406c8f0 0x7fd52406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:11.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.466+0000 7fd53e000640 1 -- 172.21.15.26:0/3851023635 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7fd53008b8f0 con 0x7fd54806e700 2024-04-26T21:10:11.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.466+0000 7fd545400640 1 --2- 172.21.15.26:0/3851023635 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd52406c8f0 0x7fd52406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:11.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.467+0000 7fd545400640 1 --2- 172.21.15.26:0/3851023635 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd52406c8f0 0x7fd52406edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd538034000 tx=0x7fd53802f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:11.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.470+0000 7fd53e000640 1 -- 172.21.15.26:0/3851023635 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd530059c20 con 0x7fd54806e700 2024-04-26T21:10:11.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.674+0000 7fd547200640 1 -- 172.21.15.26:0/3851023635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5480b5da0 con 0x7fd54806e700 2024-04-26T21:10:11.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.678+0000 7fd53e000640 1 -- 172.21.15.26:0/3851023635 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+18433 (secure 0 0 0) 0x7fd530059520 con 0x7fd54806e700 2024-04-26T21:10:11.680 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:11.680 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":37,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:10:06.822858+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:46.646256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:47.613883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-04-26T21:10:04.249730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:10:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.680+0000 7fd547200640 1 -- 172.21.15.26:0/3851023635 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd52406c8f0 msgr2=0x7fd52406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.680+0000 7fd547200640 1 --2- 172.21.15.26:0/3851023635 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd52406c8f0 0x7fd52406edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd538034000 tx=0x7fd53802f000 comp rx=0 tx=0).stop 2024-04-26T21:10:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.680+0000 7fd547200640 1 -- 172.21.15.26:0/3851023635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54806e700 msgr2=0x7fd5481606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.680+0000 7fd547200640 1 --2- 172.21.15.26:0/3851023635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54806e700 0x7fd5481606a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd53000c480 tx=0x7fd53000c940 comp rx=0 tx=0).stop 2024-04-26T21:10:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.681+0000 7fd547200640 1 -- 172.21.15.26:0/3851023635 shutdown_connections 2024-04-26T21:10:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.681+0000 7fd547200640 1 --2- 172.21.15.26:0/3851023635 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fd52406c8f0 0x7fd52406edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.681+0000 7fd547200640 1 --2- 172.21.15.26:0/3851023635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5481677a0 0x7fd54815f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.681+0000 7fd547200640 1 --2- 172.21.15.26:0/3851023635 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd54815dcb0 0x7fd54815ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.681+0000 7fd547200640 1 --2- 172.21.15.26:0/3851023635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54806e700 0x7fd5481606a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.681+0000 7fd547200640 1 -- 172.21.15.26:0/3851023635 >> 172.21.15.26:0/3851023635 conn(0x7fd54806c690 msgr2=0x7fd54815b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.681+0000 7fd547200640 1 -- 172.21.15.26:0/3851023635 shutdown_connections 2024-04-26T21:10:11.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.681+0000 7fd547200640 1 -- 172.21.15.26:0/3851023635 wait complete. 2024-04-26T21:10:11.701 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:08:31.322511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:09:46.646256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:09:47.613883+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2024-04-26T21:10:04.249730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:10:11.702 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:10:11.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.907+0000 7fb753e00640 1 Processor -- start 2024-04-26T21:10:11.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.908+0000 7fb753e00640 1 -- start start 2024-04-26T21:10:11.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.909+0000 7fb753e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7541632e0 0x7fb7540b8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:11.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.909+0000 7fb753e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7540d3870 con 0x7fb75415f290 2024-04-26T21:10:11.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.909+0000 7fb753e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7540d3b50 con 0x7fb754162e60 2024-04-26T21:10:11.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.909+0000 7fb753e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb7540d3e30 con 0x7fb7541632e0 2024-04-26T21:10:11.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.909+0000 7fb752a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb75415f290 0x7fb7540d5700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33872/0 (socket says 172.21.15.26:33872) 2024-04-26T21:10:11.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.910+0000 7fb752a00640 1 -- 172.21.15.26:0/3432448310 learned_addr learned my addr 172.21.15.26:0/3432448310 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:11.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.910+0000 7fb751600640 1 -- 172.21.15.26:0/3432448310 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3950223718 0 0) 0x7fb7540d3870 con 0x7fb75415f290 2024-04-26T21:10:11.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.910+0000 7fb751600640 1 -- 172.21.15.26:0/3432448310 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb734003520 con 0x7fb75415f290 2024-04-26T21:10:11.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.910+0000 7fb753400640 1 --2- 172.21.15.26:0/3432448310 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7541632e0 0x7fb7540b8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:11.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.910+0000 7fb751600640 1 -- 172.21.15.26:0/3432448310 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 558470736 0 0) 0x7fb734003520 con 0x7fb75415f290 2024-04-26T21:10:11.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.910+0000 7fb751600640 1 -- 172.21.15.26:0/3432448310 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7541632e0 msgr2=0x7fb7540b8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:11.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.910+0000 7fb751600640 1 --2- 172.21.15.26:0/3432448310 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7541632e0 0x7fb7540b8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:11.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.910+0000 7fb751600640 1 -- 172.21.15.26:0/3432448310 >> v1:172.21.15.86:6789/0 conn(0x7fb754162e60 legacy=0x7fb7540b8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:11.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.910+0000 7fb751600640 1 -- 172.21.15.26:0/3432448310 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7540b7e40 con 0x7fb75415f290 2024-04-26T21:10:11.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.910+0000 7fb751600640 1 -- 172.21.15.26:0/3432448310 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7fb744001690 con 0x7fb75415f290 2024-04-26T21:10:11.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.910+0000 7fb751600640 1 -- 172.21.15.26:0/3432448310 >> v1:172.21.15.26:6789/0 conn(0x7fb75415f290 legacy=0x7fb7540d5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:11.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.910+0000 7fb751600640 1 --2- 172.21.15.26:0/3432448310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7340039e0 0x7fb734003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:11.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.911+0000 7fb751600640 1 --2- 172.21.15.26:0/3432448310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb734004800 0x7fb734004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:11.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.911+0000 7fb751600640 1 --2- 172.21.15.26:0/3432448310 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7541632e0 0x7fb734005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:11.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.911+0000 7fb751600640 1 -- 172.21.15.26:0/3432448310 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb7540d3e30 con 0x7fb734004800 2024-04-26T21:10:11.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.911+0000 7fb751600640 1 -- 172.21.15.26:0/3432448310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb73400baf0 con 0x7fb7340039e0 2024-04-26T21:10:11.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.911+0000 7fb751600640 1 -- 172.21.15.26:0/3432448310 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb73400bcf0 con 0x7fb7541632e0 2024-04-26T21:10:11.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.911+0000 7fb752000640 1 --2- 172.21.15.26:0/3432448310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb734004800 0x7fb734004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:11.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.911+0000 7fb752a00640 1 --2- 172.21.15.26:0/3432448310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7340039e0 0x7fb734003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:11.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.911+0000 7fb753400640 1 --2- 172.21.15.26:0/3432448310 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7541632e0 0x7fb734005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:11.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.911+0000 7fb752000640 1 -- 172.21.15.26:0/3432448310 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7541632e0 msgr2=0x7fb734005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:11.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.911+0000 7fb752000640 1 --2- 172.21.15.26:0/3432448310 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7541632e0 0x7fb734005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:11.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.911+0000 7fb752000640 1 -- 172.21.15.26:0/3432448310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7340039e0 msgr2=0x7fb734003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:11.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.911+0000 7fb752000640 1 --2- 172.21.15.26:0/3432448310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7340039e0 0x7fb734003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:11.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.911+0000 7fb752000640 1 -- 172.21.15.26:0/3432448310 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb734003520 con 0x7fb734004800 2024-04-26T21:10:11.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.913+0000 7fb752000640 1 --2- 172.21.15.26:0/3432448310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb734004800 0x7fb734004c80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb73c001e50 tx=0x7fb734005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e9eb6813181378 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:11.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.913+0000 7fb751600640 1 -- 172.21.15.26:0/3432448310 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c001a70 con 0x7fb734004800 2024-04-26T21:10:11.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.913+0000 7fb751600640 1 -- 172.21.15.26:0/3432448310 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb73c003970 con 0x7fb734004800 2024-04-26T21:10:11.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.913+0000 7fb751600640 1 -- 172.21.15.26:0/3432448310 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c003d60 con 0x7fb734004800 2024-04-26T21:10:11.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.914+0000 7fb753e00640 1 -- 172.21.15.26:0/3432448310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb734004800 msgr2=0x7fb734004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:11.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.914+0000 7fb753e00640 1 --2- 172.21.15.26:0/3432448310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb734004800 0x7fb734004c80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb73c001e50 tx=0x7fb734005830 comp rx=0 tx=0).stop 2024-04-26T21:10:11.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.914+0000 7fb753400640 1 -- 172.21.15.26:0/3432448310 reap_dead start 2024-04-26T21:10:11.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.914+0000 7fb753e00640 1 -- 172.21.15.26:0/3432448310 shutdown_connections 2024-04-26T21:10:11.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.914+0000 7fb753e00640 1 -- 172.21.15.26:0/3432448310 >> 172.21.15.26:0/3432448310 conn(0x7fb75409e380 msgr2=0x7fb7540d4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:11.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.914+0000 7fb753e00640 1 -- 172.21.15.26:0/3432448310 shutdown_connections 2024-04-26T21:10:11.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.914+0000 7fb753e00640 1 -- 172.21.15.26:0/3432448310 wait complete. 2024-04-26T21:10:11.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.916+0000 7fb753e00640 1 Processor -- start 2024-04-26T21:10:11.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.916+0000 7fb753e00640 1 -- start start 2024-04-26T21:10:11.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.917+0000 7fb753e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb734004800 0x7fb7540cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:11.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.917+0000 7fb753e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb75415c5a0 0x7fb75414f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:11.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.917+0000 7fb753e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7540cd7d0 0x7fb7540ce440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:11.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.917+0000 7fb753e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb7540d5e10 con 0x7fb734004800 2024-04-26T21:10:11.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.917+0000 7fb753e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb754163750 con 0x7fb7540cd7d0 2024-04-26T21:10:11.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.917+0000 7fb753e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb7540d3540 con 0x7fb75415c5a0 2024-04-26T21:10:11.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.917+0000 7fb752a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb734004800 0x7fb7540cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:11.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.918+0000 7fb752a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb734004800 0x7fb7540cc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46908/0 (socket says 172.21.15.26:46908) 2024-04-26T21:10:11.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.918+0000 7fb752a00640 1 -- 172.21.15.26:0/2374391371 learned_addr learned my addr 172.21.15.26:0/2374391371 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:11.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.918+0000 7fb753400640 1 --2- 172.21.15.26:0/2374391371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7540cd7d0 0x7fb7540ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:11.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.918+0000 7fb752000640 1 --2- 172.21.15.26:0/2374391371 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb75415c5a0 0x7fb75414f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:11.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.918+0000 7fb752a00640 1 -- 172.21.15.26:0/2374391371 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb75415c5a0 msgr2=0x7fb75414f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:11.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.918+0000 7fb752a00640 1 --2- 172.21.15.26:0/2374391371 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb75415c5a0 0x7fb75414f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:11.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.918+0000 7fb752a00640 1 -- 172.21.15.26:0/2374391371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7540cd7d0 msgr2=0x7fb7540ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:11.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.918+0000 7fb752a00640 1 --2- 172.21.15.26:0/2374391371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7540cd7d0 0x7fb7540ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:11.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.918+0000 7fb752a00640 1 -- 172.21.15.26:0/2374391371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7540cc730 con 0x7fb734004800 2024-04-26T21:10:11.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.918+0000 7fb752000640 1 --2- 172.21.15.26:0/2374391371 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb75415c5a0 0x7fb75414f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:11.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.918+0000 7fb752a00640 1 --2- 172.21.15.26:0/2374391371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb734004800 0x7fb7540cc030 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb74400e7b0 tx=0x7fb74400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:11.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.919+0000 7fb74aa00640 1 -- 172.21.15.26:0/2374391371 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74400ee60 con 0x7fb734004800 2024-04-26T21:10:11.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.919+0000 7fb74aa00640 1 -- 172.21.15.26:0/2374391371 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb744018070 con 0x7fb734004800 2024-04-26T21:10:11.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.919+0000 7fb74aa00640 1 -- 172.21.15.26:0/2374391371 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74400ca90 con 0x7fb734004800 2024-04-26T21:10:11.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.919+0000 7fb753e00640 1 -- 172.21.15.26:0/2374391371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7540cca10 con 0x7fb734004800 2024-04-26T21:10:11.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.919+0000 7fb753e00640 1 -- 172.21.15.26:0/2374391371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7540ccfd0 con 0x7fb734004800 2024-04-26T21:10:11.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.920+0000 7fb753e00640 1 -- 172.21.15.26:0/2374391371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb75406a700 con 0x7fb734004800 2024-04-26T21:10:11.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.921+0000 7fb74aa00640 1 -- 172.21.15.26:0/2374391371 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb74400cc30 con 0x7fb734004800 2024-04-26T21:10:11.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.923+0000 7fb74aa00640 1 --2- 172.21.15.26:0/2374391371 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb73006c920 0x7fb73006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:11.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.923+0000 7fb74aa00640 1 -- 172.21.15.26:0/2374391371 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7fb74408b920 con 0x7fb734004800 2024-04-26T21:10:11.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.923+0000 7fb752000640 1 --2- 172.21.15.26:0/2374391371 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb73006c920 0x7fb73006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:11.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.924+0000 7fb752000640 1 --2- 172.21.15.26:0/2374391371 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb73006c920 0x7fb73006ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb73c032000 tx=0x7fb73c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:11.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:11.927+0000 7fb74aa00640 1 -- 172.21.15.26:0/2374391371 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb744059a90 con 0x7fb734004800 2024-04-26T21:10:12.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.133+0000 7fb753e00640 1 -- 172.21.15.26:0/2374391371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb7540b76e0 con 0x7fb734004800 2024-04-26T21:10:12.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.134+0000 7fb74aa00640 1 -- 172.21.15.26:0/2374391371 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) ==== 93+0+10 (secure 0 0 0) 0x7fb744059390 con 0x7fb734004800 2024-04-26T21:10:12.135 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-26T21:10:12.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.136+0000 7fb753e00640 1 -- 172.21.15.26:0/2374391371 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb73006c920 msgr2=0x7fb73006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:12.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.136+0000 7fb753e00640 1 --2- 172.21.15.26:0/2374391371 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb73006c920 0x7fb73006ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb73c032000 tx=0x7fb73c02e000 comp rx=0 tx=0).stop 2024-04-26T21:10:12.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.136+0000 7fb753e00640 1 -- 172.21.15.26:0/2374391371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb734004800 msgr2=0x7fb7540cc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:12.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.136+0000 7fb753e00640 1 --2- 172.21.15.26:0/2374391371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb734004800 0x7fb7540cc030 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb74400e7b0 tx=0x7fb74400ec70 comp rx=0 tx=0).stop 2024-04-26T21:10:12.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.136+0000 7fb753e00640 1 -- 172.21.15.26:0/2374391371 shutdown_connections 2024-04-26T21:10:12.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.136+0000 7fb753e00640 1 --2- 172.21.15.26:0/2374391371 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb73006c920 0x7fb73006ede0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:12.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.136+0000 7fb753e00640 1 --2- 172.21.15.26:0/2374391371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7540cd7d0 0x7fb7540ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:12.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.136+0000 7fb753e00640 1 --2- 172.21.15.26:0/2374391371 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb75415c5a0 0x7fb75414f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:12.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.136+0000 7fb753e00640 1 --2- 172.21.15.26:0/2374391371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb734004800 0x7fb7540cc030 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:12.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.136+0000 7fb753e00640 1 -- 172.21.15.26:0/2374391371 >> 172.21.15.26:0/2374391371 conn(0x7fb75409e380 msgr2=0x7fb7540d56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:12.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.137+0000 7fb753e00640 1 -- 172.21.15.26:0/2374391371 shutdown_connections 2024-04-26T21:10:12.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.137+0000 7fb753e00640 1 -- 172.21.15.26:0/2374391371 wait complete. 2024-04-26T21:10:12.158 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:10:12.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.374+0000 7f7d08c00640 1 Processor -- start 2024-04-26T21:10:12.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.374+0000 7f7d08c00640 1 -- start start 2024-04-26T21:10:12.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.375+0000 7f7d08c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7d0415bf30 0x7f7d04072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:12.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.375+0000 7f7d08c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7d040b4a00 0x7f7d04073240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:12.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.376+0000 7f7d08c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d0406e720 con 0x7f7d040727d0 2024-04-26T21:10:12.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.376+0000 7f7d08c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f7d0406e1f0 con 0x7f7d040b4a00 2024-04-26T21:10:12.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.376+0000 7f7d08c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7d040cf770 con 0x7f7d0415bf30 2024-04-26T21:10:12.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.376+0000 7f7d01400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7d040b4a00 0x7f7d04073240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:12.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.376+0000 7f7d02800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7d040727d0 0x7f7d0415fbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33880/0 (socket says 172.21.15.26:33880) 2024-04-26T21:10:12.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.376+0000 7f7d02800640 1 -- 172.21.15.26:0/1028708100 learned_addr learned my addr 172.21.15.26:0/1028708100 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:12.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.376+0000 7f7d01e00640 1 --2- 172.21.15.26:0/1028708100 >> v2:172.21.15.86:3300/0 conn(0x7f7d0415bf30 0x7f7d04072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:12.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.377+0000 7f7d01400640 1 -- 172.21.15.26:0/1028708100 >> v2:172.21.15.86:3300/0 conn(0x7f7d0415bf30 msgr2=0x7f7d04072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:12.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.377+0000 7f7d01400640 1 --2- 172.21.15.26:0/1028708100 >> v2:172.21.15.86:3300/0 conn(0x7f7d0415bf30 0x7f7d04072d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:12.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.377+0000 7f7d01400640 1 -- 172.21.15.26:0/1028708100 >> v1:172.21.15.26:6789/0 conn(0x7f7d040727d0 legacy=0x7f7d0415fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:12.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.377+0000 7f7d01400640 1 -- 172.21.15.26:0/1028708100 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0406e440 con 0x7f7d040b4a00 2024-04-26T21:10:12.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.377+0000 7f7d00a00640 1 -- 172.21.15.26:0/1028708100 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 833958184 0 0) 0x7f7d0406e720 con 0x7f7d040727d0 2024-04-26T21:10:12.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.377+0000 7f7d01e00640 1 --2- 172.21.15.26:0/1028708100 >> v2:172.21.15.86:3300/0 conn(0x7f7d0415bf30 0x7f7d04072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:10:12.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.378+0000 7f7d01400640 1 --2- 172.21.15.26:0/1028708100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d040b4a00 0x7f7d04073240 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7d040b3a40 tx=0x7f7cf002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caccd7487163b4f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:12.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.378+0000 7f7d00a00640 1 -- 172.21.15.26:0/1028708100 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0007740 con 0x7f7d040b4a00 2024-04-26T21:10:12.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.378+0000 7f7d00a00640 1 -- 172.21.15.26:0/1028708100 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf00078e0 con 0x7f7d040b4a00 2024-04-26T21:10:12.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.378+0000 7f7d00a00640 1 -- 172.21.15.26:0/1028708100 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0007cb0 con 0x7f7d040b4a00 2024-04-26T21:10:12.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.379+0000 7f7d08c00640 1 -- 172.21.15.26:0/1028708100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d040b4a00 msgr2=0x7f7d04073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:12.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.379+0000 7f7d08c00640 1 --2- 172.21.15.26:0/1028708100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d040b4a00 0x7f7d04073240 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7d040b3a40 tx=0x7f7cf002a5f0 comp rx=0 tx=0).stop 2024-04-26T21:10:12.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.379+0000 7f7d08c00640 1 -- 172.21.15.26:0/1028708100 shutdown_connections 2024-04-26T21:10:12.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.379+0000 7f7d08c00640 1 --2- 172.21.15.26:0/1028708100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d040b4a00 0x7f7d04073240 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:12.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.379+0000 7f7d08c00640 1 --2- 172.21.15.26:0/1028708100 >> v2:172.21.15.86:3300/0 conn(0x7f7d0415bf30 0x7f7d04072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:12.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.379+0000 7f7d08c00640 1 -- 172.21.15.26:0/1028708100 >> 172.21.15.26:0/1028708100 conn(0x7f7d040b2a80 msgr2=0x7f7d040b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:12.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.379+0000 7f7d08c00640 1 -- 172.21.15.26:0/1028708100 shutdown_connections 2024-04-26T21:10:12.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.379+0000 7f7d08c00640 1 -- 172.21.15.26:0/1028708100 wait complete. 2024-04-26T21:10:12.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.381+0000 7f7d08c00640 1 Processor -- start 2024-04-26T21:10:12.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.382+0000 7f7d08c00640 1 -- start start 2024-04-26T21:10:12.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.383+0000 7f7d08c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d040727d0 0x7f7d0415e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:12.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.383+0000 7f7d08c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d040b4a00 0x7f7d04152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:12.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.383+0000 7f7d08c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d0415bf30 0x7f7d04152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:12.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.383+0000 7f7d08c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0415c490 con 0x7f7d0415bf30 2024-04-26T21:10:12.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.383+0000 7f7d08c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0415c310 con 0x7f7d040727d0 2024-04-26T21:10:12.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.383+0000 7f7d08c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7d04073a80 con 0x7f7d040b4a00 2024-04-26T21:10:12.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.383+0000 7f7d02800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d0415bf30 0x7f7d04152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:12.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.383+0000 7f7d02800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d0415bf30 0x7f7d04152690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46926/0 (socket says 172.21.15.26:46926) 2024-04-26T21:10:12.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.383+0000 7f7d02800640 1 -- 172.21.15.26:0/1790850646 learned_addr learned my addr 172.21.15.26:0/1790850646 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:12.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.384+0000 7f7d01400640 1 --2- 172.21.15.26:0/1790850646 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d040b4a00 0x7f7d04152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:12.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.384+0000 7f7d02800640 1 -- 172.21.15.26:0/1790850646 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d040b4a00 msgr2=0x7f7d04152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:12.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.384+0000 7f7d02800640 1 --2- 172.21.15.26:0/1790850646 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d040b4a00 0x7f7d04152150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:12.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.384+0000 7f7d02800640 1 -- 172.21.15.26:0/1790850646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d040727d0 msgr2=0x7f7d0415e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:12.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.384+0000 7f7d02800640 1 --2- 172.21.15.26:0/1790850646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d040727d0 0x7f7d0415e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:12.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.384+0000 7f7d02800640 1 -- 172.21.15.26:0/1790850646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d041f5240 con 0x7f7d0415bf30 2024-04-26T21:10:12.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.384+0000 7f7d02800640 1 --2- 172.21.15.26:0/1790850646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d0415bf30 0x7f7d04152690 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00c450 tx=0x7f7cfc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:12.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.384+0000 7f7cfa000640 1 -- 172.21.15.26:0/1790850646 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc018070 con 0x7f7d0415bf30 2024-04-26T21:10:12.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.384+0000 7f7cfa000640 1 -- 172.21.15.26:0/1790850646 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cfc007500 con 0x7f7d0415bf30 2024-04-26T21:10:12.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.384+0000 7f7cfa000640 1 -- 172.21.15.26:0/1790850646 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc004a20 con 0x7f7d0415bf30 2024-04-26T21:10:12.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.385+0000 7f7d08c00640 1 -- 172.21.15.26:0/1790850646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d041f5520 con 0x7f7d0415bf30 2024-04-26T21:10:12.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.385+0000 7f7d08c00640 1 -- 172.21.15.26:0/1790850646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d041f5ae0 con 0x7f7d0415bf30 2024-04-26T21:10:12.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.386+0000 7f7d08c00640 1 -- 172.21.15.26:0/1790850646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d0406a700 con 0x7f7d0415bf30 2024-04-26T21:10:12.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.387+0000 7f7cfa000640 1 -- 172.21.15.26:0/1790850646 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7cfc004bc0 con 0x7f7d0415bf30 2024-04-26T21:10:12.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.388+0000 7f7cfa000640 1 --2- 172.21.15.26:0/1790850646 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f7ce406c8f0 0x7f7ce406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:12.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.388+0000 7f7cfa000640 1 -- 172.21.15.26:0/1790850646 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f7cfc08af80 con 0x7f7d0415bf30 2024-04-26T21:10:12.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.389+0000 7f7d01e00640 1 --2- 172.21.15.26:0/1790850646 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f7ce406c8f0 0x7f7ce406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:12.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.389+0000 7f7d01e00640 1 --2- 172.21.15.26:0/1790850646 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f7ce406c8f0 0x7f7ce406edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4006fd0 tx=0x7f7cf4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:12.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.393+0000 7f7cfa000640 1 -- 172.21.15.26:0/1790850646 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7cfc059210 con 0x7f7d0415bf30 2024-04-26T21:10:12.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.597+0000 7f7d08c00640 1 -- 172.21.15.26:0/1790850646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7d0406fe30 con 0x7f7d0415bf30 2024-04-26T21:10:12.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.598+0000 7f7cfa000640 1 -- 172.21.15.26:0/1790850646 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) ==== 104+0+11 (secure 0 0 0) 0x7f7cfc058b10 con 0x7f7d0415bf30 2024-04-26T21:10:12.600 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-26T21:10:12.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.601+0000 7f7d08c00640 1 -- 172.21.15.26:0/1790850646 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f7ce406c8f0 msgr2=0x7f7ce406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:12.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.601+0000 7f7d08c00640 1 --2- 172.21.15.26:0/1790850646 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f7ce406c8f0 0x7f7ce406edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4006fd0 tx=0x7f7cf4008040 comp rx=0 tx=0).stop 2024-04-26T21:10:12.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.601+0000 7f7d08c00640 1 -- 172.21.15.26:0/1790850646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d0415bf30 msgr2=0x7f7d04152690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:12.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.601+0000 7f7d08c00640 1 --2- 172.21.15.26:0/1790850646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d0415bf30 0x7f7d04152690 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00c450 tx=0x7f7cfc00c910 comp rx=0 tx=0).stop 2024-04-26T21:10:12.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.601+0000 7f7d08c00640 1 -- 172.21.15.26:0/1790850646 shutdown_connections 2024-04-26T21:10:12.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.601+0000 7f7d08c00640 1 --2- 172.21.15.26:0/1790850646 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f7ce406c8f0 0x7f7ce406edb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:12.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.601+0000 7f7d08c00640 1 --2- 172.21.15.26:0/1790850646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d0415bf30 0x7f7d04152690 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:12.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.601+0000 7f7d08c00640 1 --2- 172.21.15.26:0/1790850646 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d040b4a00 0x7f7d04152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:12.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.601+0000 7f7d08c00640 1 --2- 172.21.15.26:0/1790850646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d040727d0 0x7f7d0415e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:12.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.601+0000 7f7d08c00640 1 -- 172.21.15.26:0/1790850646 >> 172.21.15.26:0/1790850646 conn(0x7f7d040b2a80 msgr2=0x7f7d04156890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:12.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.601+0000 7f7d08c00640 1 -- 172.21.15.26:0/1790850646 shutdown_connections 2024-04-26T21:10:12.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.601+0000 7f7d08c00640 1 -- 172.21.15.26:0/1790850646 wait complete. 2024-04-26T21:10:12.624 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:10:12.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.836+0000 7f43a0a00640 1 Processor -- start 2024-04-26T21:10:12.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.836+0000 7f43a0a00640 1 -- start start 2024-04-26T21:10:12.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.838+0000 7f43a0a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f439c146a30 0x7f439c0b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:12.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.838+0000 7f43a0a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f439c06f9f0 0x7f439c0cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:12.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.838+0000 7f43a0a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f439c0b42c0 con 0x7f439c15d590 2024-04-26T21:10:12.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.838+0000 7f43a0a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f439c0cebd0 con 0x7f439c146a30 2024-04-26T21:10:12.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.838+0000 7f43a0a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f439c0ceeb0 con 0x7f439c06f9f0 2024-04-26T21:10:12.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.838+0000 7f439a800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f439c146a30 0x7f439c0b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:12.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.838+0000 7f439a800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f439c146a30 0x7f439c0b3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46942/0 (socket says 172.21.15.26:46942) 2024-04-26T21:10:12.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.838+0000 7f439a800640 1 -- 172.21.15.26:0/1399369709 learned_addr learned my addr 172.21.15.26:0/1399369709 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:12.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.839+0000 7f439b200640 1 --2- 172.21.15.26:0/1399369709 >> v2:172.21.15.86:3300/0 conn(0x7f439c06f9f0 0x7f439c0cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:12.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.839+0000 7f439a800640 1 -- 172.21.15.26:0/1399369709 >> v2:172.21.15.86:3300/0 conn(0x7f439c06f9f0 msgr2=0x7f439c0cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:12.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.839+0000 7f439a800640 1 --2- 172.21.15.26:0/1399369709 >> v2:172.21.15.86:3300/0 conn(0x7f439c06f9f0 0x7f439c0cff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:12.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.839+0000 7f439a800640 1 -- 172.21.15.26:0/1399369709 >> v1:172.21.15.86:6789/0 conn(0x7f439c15d590 legacy=0x7f439c06f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:12.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.839+0000 7f439a800640 1 -- 172.21.15.26:0/1399369709 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439c0cf190 con 0x7f439c146a30 2024-04-26T21:10:12.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.840+0000 7f439a800640 1 --2- 172.21.15.26:0/1399369709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f439c146a30 0x7f439c0b3d80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f439c06daa0 tx=0x7f438c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fb411936f34449 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:12.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.840+0000 7f4399400640 1 -- 172.21.15.26:0/1399369709 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c004030 con 0x7f439c146a30 2024-04-26T21:10:12.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.840+0000 7f4399400640 1 -- 172.21.15.26:0/1399369709 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f438c0041d0 con 0x7f439c146a30 2024-04-26T21:10:12.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.840+0000 7f4399400640 1 -- 172.21.15.26:0/1399369709 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c003610 con 0x7f439c146a30 2024-04-26T21:10:12.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.841+0000 7f43a0a00640 1 -- 172.21.15.26:0/1399369709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f439c146a30 msgr2=0x7f439c0b3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:12.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.841+0000 7f43a0a00640 1 --2- 172.21.15.26:0/1399369709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f439c146a30 0x7f439c0b3d80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f439c06daa0 tx=0x7f438c02a200 comp rx=0 tx=0).stop 2024-04-26T21:10:12.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.841+0000 7f43a0a00640 1 -- 172.21.15.26:0/1399369709 shutdown_connections 2024-04-26T21:10:12.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.841+0000 7f43a0a00640 1 --2- 172.21.15.26:0/1399369709 >> v2:172.21.15.86:3300/0 conn(0x7f439c06f9f0 0x7f439c0cff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:12.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.841+0000 7f43a0a00640 1 --2- 172.21.15.26:0/1399369709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f439c146a30 0x7f439c0b3d80 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:12.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.841+0000 7f43a0a00640 1 -- 172.21.15.26:0/1399369709 >> 172.21.15.26:0/1399369709 conn(0x7f439c0cd7d0 msgr2=0x7f439c0cdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:12.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.841+0000 7f43a0a00640 1 -- 172.21.15.26:0/1399369709 shutdown_connections 2024-04-26T21:10:12.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.841+0000 7f43a0a00640 1 -- 172.21.15.26:0/1399369709 wait complete. 2024-04-26T21:10:12.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.843+0000 7f43a0a00640 1 Processor -- start 2024-04-26T21:10:12.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.843+0000 7f43a0a00640 1 -- start start 2024-04-26T21:10:12.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.844+0000 7f43a0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f439c06f9f0 0x7f439c15c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:12.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.844+0000 7f43a0a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f439c146a30 0x7f439c143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:12.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.844+0000 7f43a0a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f439c15d590 0x7f439c143b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:12.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.844+0000 7f43a0a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f439c0cfa30 con 0x7f439c146a30 2024-04-26T21:10:12.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.844+0000 7f43a0a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f439c0cf750 con 0x7f439c06f9f0 2024-04-26T21:10:12.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.844+0000 7f43a0a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f439c0b6d20 con 0x7f439c15d590 2024-04-26T21:10:12.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.844+0000 7f4399e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f439c146a30 0x7f439c143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:12.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.844+0000 7f4399e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f439c146a30 0x7f439c143650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46952/0 (socket says 172.21.15.26:46952) 2024-04-26T21:10:12.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.844+0000 7f4399e00640 1 -- 172.21.15.26:0/3834541931 learned_addr learned my addr 172.21.15.26:0/3834541931 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:12.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.844+0000 7f439a800640 1 --2- 172.21.15.26:0/3834541931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f439c06f9f0 0x7f439c15c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:12.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.844+0000 7f439b200640 1 --2- 172.21.15.26:0/3834541931 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f439c15d590 0x7f439c143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:12.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.845+0000 7f4399e00640 1 -- 172.21.15.26:0/3834541931 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f439c15d590 msgr2=0x7f439c143b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:12.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.845+0000 7f4399e00640 1 --2- 172.21.15.26:0/3834541931 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f439c15d590 0x7f439c143b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:12.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.845+0000 7f4399e00640 1 -- 172.21.15.26:0/3834541931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f439c06f9f0 msgr2=0x7f439c15c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:12.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.845+0000 7f4399e00640 1 --2- 172.21.15.26:0/3834541931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f439c06f9f0 0x7f439c15c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:12.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.845+0000 7f4399e00640 1 -- 172.21.15.26:0/3834541931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439c15cd70 con 0x7f439c146a30 2024-04-26T21:10:12.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.845+0000 7f4399e00640 1 --2- 172.21.15.26:0/3834541931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f439c146a30 0x7f439c143650 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f438800b410 tx=0x7f438800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:12.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.845+0000 7f4392a00640 1 -- 172.21.15.26:0/3834541931 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4388015020 con 0x7f439c146a30 2024-04-26T21:10:12.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.845+0000 7f4392a00640 1 -- 172.21.15.26:0/3834541931 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4388011830 con 0x7f439c146a30 2024-04-26T21:10:12.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.845+0000 7f4392a00640 1 -- 172.21.15.26:0/3834541931 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4388011c20 con 0x7f439c146a30 2024-04-26T21:10:12.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.845+0000 7f43a0a00640 1 -- 172.21.15.26:0/3834541931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f439c138820 con 0x7f439c146a30 2024-04-26T21:10:12.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.845+0000 7f43a0a00640 1 -- 172.21.15.26:0/3834541931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f439c1390c0 con 0x7f439c146a30 2024-04-26T21:10:12.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.847+0000 7f43a0a00640 1 -- 172.21.15.26:0/3834541931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f439c0506c0 con 0x7f439c146a30 2024-04-26T21:10:12.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.848+0000 7f4392a00640 1 -- 172.21.15.26:0/3834541931 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4388018020 con 0x7f439c146a30 2024-04-26T21:10:12.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.849+0000 7f4392a00640 1 --2- 172.21.15.26:0/3834541931 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f437c06c8f0 0x7f437c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:12.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.849+0000 7f4392a00640 1 -- 172.21.15.26:0/3834541931 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f438808bd30 con 0x7f439c146a30 2024-04-26T21:10:12.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.849+0000 7f439a800640 1 --2- 172.21.15.26:0/3834541931 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f437c06c8f0 0x7f437c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:12.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.850+0000 7f439a800640 1 --2- 172.21.15.26:0/3834541931 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f437c06c8f0 0x7f437c06edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f438c033000 tx=0x7f438c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:12.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:12.854+0000 7f4392a00640 1 -- 172.21.15.26:0/3834541931 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f438801f020 con 0x7f439c146a30 2024-04-26T21:10:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.058+0000 7f43a0a00640 1 -- 172.21.15.26:0/3834541931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f439c13e6b0 con 0x7f439c146a30 2024-04-26T21:10:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.059+0000 7f4392a00640 1 -- 172.21.15.26:0/3834541931 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) ==== 100+0+11 (secure 0 0 0) 0x7f43880172c0 con 0x7f439c146a30 2024-04-26T21:10:13.061 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-26T21:10:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.062+0000 7f43a0a00640 1 -- 172.21.15.26:0/3834541931 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f437c06c8f0 msgr2=0x7f437c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.062+0000 7f43a0a00640 1 --2- 172.21.15.26:0/3834541931 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f437c06c8f0 0x7f437c06edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f438c033000 tx=0x7f438c02e000 comp rx=0 tx=0).stop 2024-04-26T21:10:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.062+0000 7f43a0a00640 1 -- 172.21.15.26:0/3834541931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f439c146a30 msgr2=0x7f439c143650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.062+0000 7f43a0a00640 1 --2- 172.21.15.26:0/3834541931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f439c146a30 0x7f439c143650 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f438800b410 tx=0x7f438800b8d0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.062+0000 7f43a0a00640 1 -- 172.21.15.26:0/3834541931 shutdown_connections 2024-04-26T21:10:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.062+0000 7f43a0a00640 1 --2- 172.21.15.26:0/3834541931 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f437c06c8f0 0x7f437c06edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.062+0000 7f43a0a00640 1 --2- 172.21.15.26:0/3834541931 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f439c15d590 0x7f439c143b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.062+0000 7f43a0a00640 1 --2- 172.21.15.26:0/3834541931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f439c146a30 0x7f439c143650 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.062+0000 7f43a0a00640 1 --2- 172.21.15.26:0/3834541931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f439c06f9f0 0x7f439c15c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.062+0000 7f43a0a00640 1 -- 172.21.15.26:0/3834541931 >> 172.21.15.26:0/3834541931 conn(0x7f439c0cd7d0 msgr2=0x7f439c0b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.062+0000 7f43a0a00640 1 -- 172.21.15.26:0/3834541931 shutdown_connections 2024-04-26T21:10:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.062+0000 7f43a0a00640 1 -- 172.21.15.26:0/3834541931 wait complete. 2024-04-26T21:10:13.084 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-04-26T21:10:13.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.288+0000 7f5c9fe00640 1 Processor -- start 2024-04-26T21:10:13.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.289+0000 7f5c9fe00640 1 -- start start 2024-04-26T21:10:13.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.290+0000 7f5c9fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5ca015bdc0 0x7f5ca00b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:13.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.290+0000 7f5c9fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5ca0073530 0x7f5ca00cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:13.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.290+0000 7f5c9fe00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ca006fbb0 con 0x7f5ca006e210 2024-04-26T21:10:13.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.290+0000 7f5c9fe00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f5ca00b4cf0 con 0x7f5ca0073530 2024-04-26T21:10:13.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.290+0000 7f5c9fe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5ca00cf280 con 0x7f5ca015bdc0 2024-04-26T21:10:13.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.291+0000 7f5c9e000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5ca006e210 0x7f5ca0072e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33896/0 (socket says 172.21.15.26:33896) 2024-04-26T21:10:13.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.291+0000 7f5c9e000640 1 -- 172.21.15.26:0/3471211021 learned_addr learned my addr 172.21.15.26:0/3471211021 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:13.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.291+0000 7f5c9f400640 1 --2- 172.21.15.26:0/3471211021 >> v2:172.21.15.26:3300/0 conn(0x7f5ca0073530 0x7f5ca00cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:13.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.291+0000 7f5c9ea00640 1 --2- 172.21.15.26:0/3471211021 >> v2:172.21.15.86:3300/0 conn(0x7f5ca015bdc0 0x7f5ca00b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:13.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.291+0000 7f5c9f400640 1 -- 172.21.15.26:0/3471211021 >> v2:172.21.15.86:3300/0 conn(0x7f5ca015bdc0 msgr2=0x7f5ca00b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:13.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.291+0000 7f5c9f400640 1 --2- 172.21.15.26:0/3471211021 >> v2:172.21.15.86:3300/0 conn(0x7f5ca015bdc0 0x7f5ca00b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.291+0000 7f5c9d600640 1 -- 172.21.15.26:0/3471211021 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1259007849 0 0) 0x7f5ca006fbb0 con 0x7f5ca006e210 2024-04-26T21:10:13.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.291+0000 7f5c9f400640 1 -- 172.21.15.26:0/3471211021 >> v1:172.21.15.26:6789/0 conn(0x7f5ca006e210 legacy=0x7f5ca0072e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:13.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.291+0000 7f5c9f400640 1 -- 172.21.15.26:0/3471211021 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca006e780 con 0x7f5ca0073530 2024-04-26T21:10:13.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.292+0000 7f5c9f400640 1 --2- 172.21.15.26:0/3471211021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ca0073530 0x7f5ca00cf710 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00cf400 tx=0x7f5c9802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ca46e2dac220c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:13.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.293+0000 7f5c9d600640 1 -- 172.21.15.26:0/3471211021 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98037070 con 0x7f5ca0073530 2024-04-26T21:10:13.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.293+0000 7f5c9d600640 1 -- 172.21.15.26:0/3471211021 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c9802ae50 con 0x7f5ca0073530 2024-04-26T21:10:13.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.293+0000 7f5c9d600640 1 -- 172.21.15.26:0/3471211021 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98032660 con 0x7f5ca0073530 2024-04-26T21:10:13.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.293+0000 7f5c9fe00640 1 -- 172.21.15.26:0/3471211021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ca0073530 msgr2=0x7f5ca00cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:13.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.293+0000 7f5c9fe00640 1 --2- 172.21.15.26:0/3471211021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ca0073530 0x7f5ca00cf710 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00cf400 tx=0x7f5c9802a100 comp rx=0 tx=0).stop 2024-04-26T21:10:13.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.294+0000 7f5c9fe00640 1 -- 172.21.15.26:0/3471211021 shutdown_connections 2024-04-26T21:10:13.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.294+0000 7f5c9fe00640 1 --2- 172.21.15.26:0/3471211021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ca0073530 0x7f5ca00cf710 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.294+0000 7f5c9fe00640 1 --2- 172.21.15.26:0/3471211021 >> v2:172.21.15.86:3300/0 conn(0x7f5ca015bdc0 0x7f5ca00b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.294+0000 7f5c9fe00640 1 -- 172.21.15.26:0/3471211021 >> 172.21.15.26:0/3471211021 conn(0x7f5ca00b2a10 msgr2=0x7f5ca00b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:13.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.294+0000 7f5c9fe00640 1 -- 172.21.15.26:0/3471211021 shutdown_connections 2024-04-26T21:10:13.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.294+0000 7f5c9fe00640 1 -- 172.21.15.26:0/3471211021 wait complete. 2024-04-26T21:10:13.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.295+0000 7f5c9fe00640 1 Processor -- start 2024-04-26T21:10:13.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.296+0000 7f5c9fe00640 1 -- start start 2024-04-26T21:10:13.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.296+0000 7f5c9fe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ca006e210 0x7f5ca0156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:13.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.296+0000 7f5c9fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ca0073530 0x7f5ca014b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:13.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.297+0000 7f5c9fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca015bdc0 0x7f5ca00af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:13.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.297+0000 7f5c9fe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca0072380 con 0x7f5ca0073530 2024-04-26T21:10:13.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.297+0000 7f5c9fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca0070130 con 0x7f5ca015bdc0 2024-04-26T21:10:13.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.297+0000 7f5c9fe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca01f9470 con 0x7f5ca006e210 2024-04-26T21:10:13.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.297+0000 7f5c9e000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ca0073530 0x7f5ca014b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:13.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.297+0000 7f5c9e000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ca0073530 0x7f5ca014b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46984/0 (socket says 172.21.15.26:46984) 2024-04-26T21:10:13.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.297+0000 7f5c9e000640 1 -- 172.21.15.26:0/4095753753 learned_addr learned my addr 172.21.15.26:0/4095753753 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:13.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.297+0000 7f5c9ea00640 1 --2- 172.21.15.26:0/4095753753 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ca006e210 0x7f5ca0156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:13.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.297+0000 7f5c9e000640 1 -- 172.21.15.26:0/4095753753 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ca006e210 msgr2=0x7f5ca0156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:13.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.297+0000 7f5c9f400640 1 --2- 172.21.15.26:0/4095753753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca015bdc0 0x7f5ca00af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:13.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.297+0000 7f5c9e000640 1 --2- 172.21.15.26:0/4095753753 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ca006e210 0x7f5ca0156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.297+0000 7f5c9e000640 1 -- 172.21.15.26:0/4095753753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca015bdc0 msgr2=0x7f5ca00af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:13.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.297+0000 7f5c9e000640 1 --2- 172.21.15.26:0/4095753753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca015bdc0 0x7f5ca00af8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.297+0000 7f5c9e000640 1 -- 172.21.15.26:0/4095753753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca01f86b0 con 0x7f5ca0073530 2024-04-26T21:10:13.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.298+0000 7f5c9e000640 1 --2- 172.21.15.26:0/4095753753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ca0073530 0x7f5ca014b870 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00c4d0 tx=0x7f5c8c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:13.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.298+0000 7f5c96a00640 1 -- 172.21.15.26:0/4095753753 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c018070 con 0x7f5ca0073530 2024-04-26T21:10:13.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.298+0000 7f5c96a00640 1 -- 172.21.15.26:0/4095753753 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c8c007500 con 0x7f5ca0073530 2024-04-26T21:10:13.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.298+0000 7f5c96a00640 1 -- 172.21.15.26:0/4095753753 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c004a20 con 0x7f5ca0073530 2024-04-26T21:10:13.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.298+0000 7f5c9fe00640 1 -- 172.21.15.26:0/4095753753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca01f8990 con 0x7f5ca0073530 2024-04-26T21:10:13.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.298+0000 7f5c9fe00640 1 -- 172.21.15.26:0/4095753753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca01f8f50 con 0x7f5ca0073530 2024-04-26T21:10:13.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.300+0000 7f5c9fe00640 1 -- 172.21.15.26:0/4095753753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca01f9150 con 0x7f5ca0073530 2024-04-26T21:10:13.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.301+0000 7f5c96a00640 1 -- 172.21.15.26:0/4095753753 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c8c004bc0 con 0x7f5ca0073530 2024-04-26T21:10:13.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.301+0000 7f5c96a00640 1 --2- 172.21.15.26:0/4095753753 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5c8006c920 0x7f5c8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:13.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.302+0000 7f5c96a00640 1 -- 172.21.15.26:0/4095753753 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f5c8c08aea0 con 0x7f5ca0073530 2024-04-26T21:10:13.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.302+0000 7f5c9ea00640 1 --2- 172.21.15.26:0/4095753753 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5c8006c920 0x7f5c8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:13.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.303+0000 7f5c9ea00640 1 --2- 172.21.15.26:0/4095753753 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5c8006c920 0x7f5c8006ede0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5c900099c0 tx=0x7f5c90003990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:13.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.307+0000 7f5c96a00640 1 -- 172.21.15.26:0/4095753753 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5c8c0591d0 con 0x7f5ca0073530 2024-04-26T21:10:13.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.509+0000 7f5c9fe00640 1 -- 172.21.15.26:0/4095753753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) -- 0x7f5ca006a700 con 0x7f5ca0073530 2024-04-26T21:10:13.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.510+0000 7f5c96a00640 1 -- 172.21.15.26:0/4095753753 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v37) ==== 97+0+11 (secure 0 0 0) 0x7f5c8c058ad0 con 0x7f5ca0073530 2024-04-26T21:10:13.512 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-26T21:10:13.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.512+0000 7f5c9fe00640 1 -- 172.21.15.26:0/4095753753 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5c8006c920 msgr2=0x7f5c8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:13.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.512+0000 7f5c9fe00640 1 --2- 172.21.15.26:0/4095753753 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5c8006c920 0x7f5c8006ede0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5c900099c0 tx=0x7f5c90003990 comp rx=0 tx=0).stop 2024-04-26T21:10:13.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.512+0000 7f5c9fe00640 1 -- 172.21.15.26:0/4095753753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ca0073530 msgr2=0x7f5ca014b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:13.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.512+0000 7f5c9fe00640 1 --2- 172.21.15.26:0/4095753753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ca0073530 0x7f5ca014b870 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00c4d0 tx=0x7f5c8c00c990 comp rx=0 tx=0).stop 2024-04-26T21:10:13.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.513+0000 7f5c9fe00640 1 -- 172.21.15.26:0/4095753753 shutdown_connections 2024-04-26T21:10:13.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.513+0000 7f5c9fe00640 1 --2- 172.21.15.26:0/4095753753 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f5c8006c920 0x7f5c8006ede0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.513+0000 7f5c9fe00640 1 --2- 172.21.15.26:0/4095753753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ca015bdc0 0x7f5ca00af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.513+0000 7f5c9fe00640 1 --2- 172.21.15.26:0/4095753753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ca0073530 0x7f5ca014b870 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.513+0000 7f5c9fe00640 1 --2- 172.21.15.26:0/4095753753 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ca006e210 0x7f5ca0156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.513+0000 7f5c9fe00640 1 -- 172.21.15.26:0/4095753753 >> 172.21.15.26:0/4095753753 conn(0x7f5ca00b2a10 msgr2=0x7f5ca00b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:13.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.513+0000 7f5c9fe00640 1 -- 172.21.15.26:0/4095753753 shutdown_connections 2024-04-26T21:10:13.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.513+0000 7f5c9fe00640 1 -- 172.21.15.26:0/4095753753 wait complete. 2024-04-26T21:10:13.534 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:10:13.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.748+0000 7f87a2000640 1 Processor -- start 2024-04-26T21:10:13.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.748+0000 7f87a2000640 1 -- start start 2024-04-26T21:10:13.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.750+0000 7f87a2000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f879c15dcb0 0x7f879c164590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:13.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.750+0000 7f87a2000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f879c06e700 0x7f879c06ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:13.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.750+0000 7f87a2000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f879c164ad0 con 0x7f879c1677a0 2024-04-26T21:10:13.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.750+0000 7f87a2000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f879c15e090 con 0x7f879c15dcb0 2024-04-26T21:10:13.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.750+0000 7f87a2000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f879c072500 con 0x7f879c06e700 2024-04-26T21:10:13.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.750+0000 7f87a0c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f879c15dcb0 0x7f879c164590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:13.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.750+0000 7f87a1600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f879c1677a0 0x7f879c0d1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33904/0 (socket says 172.21.15.26:33904) 2024-04-26T21:10:13.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.750+0000 7f87a1600640 1 -- 172.21.15.26:0/3656117544 learned_addr learned my addr 172.21.15.26:0/3656117544 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:13.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.750+0000 7f879be00640 1 --2- 172.21.15.26:0/3656117544 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f879c06e700 0x7f879c06ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:13.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.751+0000 7f87a0c00640 1 -- 172.21.15.26:0/3656117544 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f879c06e700 msgr2=0x7f879c06ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:13.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.751+0000 7f87a0c00640 1 --2- 172.21.15.26:0/3656117544 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f879c06e700 0x7f879c06ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.751+0000 7f87a0c00640 1 -- 172.21.15.26:0/3656117544 >> v1:172.21.15.26:6789/0 conn(0x7f879c1677a0 legacy=0x7f879c0d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:13.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.751+0000 7f87a0c00640 1 -- 172.21.15.26:0/3656117544 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879c0727e0 con 0x7f879c15dcb0 2024-04-26T21:10:13.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.751+0000 7f879b400640 1 -- 172.21.15.26:0/3656117544 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2157618460 0 0) 0x7f879c164ad0 con 0x7f879c1677a0 2024-04-26T21:10:13.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.752+0000 7f87a0c00640 1 --2- 172.21.15.26:0/3656117544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f879c15dcb0 0x7f879c164590 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f878c0097a0 tx=0x7f878c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ff436d874d3b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:13.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.752+0000 7f879b400640 1 -- 172.21.15.26:0/3656117544 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c037070 con 0x7f879c15dcb0 2024-04-26T21:10:13.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.752+0000 7f879b400640 1 -- 172.21.15.26:0/3656117544 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f878c02adb0 con 0x7f879c15dcb0 2024-04-26T21:10:13.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.752+0000 7f879b400640 1 -- 172.21.15.26:0/3656117544 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c032660 con 0x7f879c15dcb0 2024-04-26T21:10:13.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.753+0000 7f87a2000640 1 -- 172.21.15.26:0/3656117544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f879c15dcb0 msgr2=0x7f879c164590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:13.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.753+0000 7f87a2000640 1 --2- 172.21.15.26:0/3656117544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f879c15dcb0 0x7f879c164590 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f878c0097a0 tx=0x7f878c02a060 comp rx=0 tx=0).stop 2024-04-26T21:10:13.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.753+0000 7f87a2000640 1 -- 172.21.15.26:0/3656117544 shutdown_connections 2024-04-26T21:10:13.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.753+0000 7f87a2000640 1 --2- 172.21.15.26:0/3656117544 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f879c06e700 0x7f879c06ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.753+0000 7f87a2000640 1 --2- 172.21.15.26:0/3656117544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f879c15dcb0 0x7f879c164590 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.753+0000 7f87a2000640 1 -- 172.21.15.26:0/3656117544 >> 172.21.15.26:0/3656117544 conn(0x7f879c06c690 msgr2=0x7f879c06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:13.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.753+0000 7f87a2000640 1 -- 172.21.15.26:0/3656117544 shutdown_connections 2024-04-26T21:10:13.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.753+0000 7f87a2000640 1 -- 172.21.15.26:0/3656117544 wait complete. 2024-04-26T21:10:13.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.755+0000 7f87a2000640 1 Processor -- start 2024-04-26T21:10:13.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.755+0000 7f87a2000640 1 -- start start 2024-04-26T21:10:13.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.757+0000 7f87a2000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f879c06e700 0x7f879c0afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:13.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.757+0000 7f87a2000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f879c15dcb0 0x7f879c15b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:13.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.757+0000 7f87a2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f879c1677a0 0x7f879c1f93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:13.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.757+0000 7f87a2000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f879c072da0 con 0x7f879c06e700 2024-04-26T21:10:13.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.757+0000 7f87a2000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f879c167280 con 0x7f879c1677a0 2024-04-26T21:10:13.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.757+0000 7f87a2000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f879c06e3c0 con 0x7f879c15dcb0 2024-04-26T21:10:13.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.757+0000 7f87a0c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f879c06e700 0x7f879c0afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:13.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.757+0000 7f87a0c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f879c06e700 0x7f879c0afec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47008/0 (socket says 172.21.15.26:47008) 2024-04-26T21:10:13.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.757+0000 7f87a0c00640 1 -- 172.21.15.26:0/3757601831 learned_addr learned my addr 172.21.15.26:0/3757601831 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:13.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.758+0000 7f87a0c00640 1 -- 172.21.15.26:0/3757601831 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f879c15dcb0 msgr2=0x7f879c15b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:13.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.758+0000 7f87a0c00640 1 --2- 172.21.15.26:0/3757601831 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f879c15dcb0 0x7f879c15b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.758+0000 7f87a1600640 1 --2- 172.21.15.26:0/3757601831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f879c1677a0 0x7f879c1f93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:13.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.758+0000 7f87a0c00640 1 -- 172.21.15.26:0/3757601831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f879c1677a0 msgr2=0x7f879c1f93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:13.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.758+0000 7f87a0c00640 1 --2- 172.21.15.26:0/3757601831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f879c1677a0 0x7f879c1f93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.758+0000 7f87a0c00640 1 -- 172.21.15.26:0/3757601831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879c15bb70 con 0x7f879c06e700 2024-04-26T21:10:13.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.758+0000 7f87a0c00640 1 --2- 172.21.15.26:0/3757601831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f879c06e700 0x7f879c0afec0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f878c0097a0 tx=0x7f878c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:13.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.758+0000 7f8798c00640 1 -- 172.21.15.26:0/3757601831 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c037040 con 0x7f879c06e700 2024-04-26T21:10:13.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.758+0000 7f8798c00640 1 -- 172.21.15.26:0/3757601831 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f878c004250 con 0x7f879c06e700 2024-04-26T21:10:13.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.758+0000 7f87a2000640 1 -- 172.21.15.26:0/3757601831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f879c1ecac0 con 0x7f879c06e700 2024-04-26T21:10:13.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.758+0000 7f8798c00640 1 -- 172.21.15.26:0/3757601831 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c003610 con 0x7f879c06e700 2024-04-26T21:10:13.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.759+0000 7f87a2000640 1 -- 172.21.15.26:0/3757601831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f879c1ed080 con 0x7f879c06e700 2024-04-26T21:10:13.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.760+0000 7f87a2000640 1 -- 172.21.15.26:0/3757601831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f879c06a700 con 0x7f879c06e700 2024-04-26T21:10:13.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.761+0000 7f8798c00640 1 -- 172.21.15.26:0/3757601831 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f878c02ac70 con 0x7f879c06e700 2024-04-26T21:10:13.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.762+0000 7f8798c00640 1 --2- 172.21.15.26:0/3757601831 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f877c06c8f0 0x7f877c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:13.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.762+0000 7f8798c00640 1 -- 172.21.15.26:0/3757601831 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f878c0acce0 con 0x7f879c06e700 2024-04-26T21:10:13.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.762+0000 7f879be00640 1 --2- 172.21.15.26:0/3757601831 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f877c06c8f0 0x7f877c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:13.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.763+0000 7f879be00640 1 --2- 172.21.15.26:0/3757601831 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f877c06c8f0 0x7f877c06edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f879c162f30 tx=0x7f8788008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.767+0000 7f8798c00640 1 -- 172.21.15.26:0/3757601831 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f879c15bb70 con 0x7f879c06e700 2024-04-26T21:10:13.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.991+0000 7f87a2000640 1 -- 172.21.15.26:0/3757601831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f879c165900 con 0x7f879c06e700 2024-04-26T21:10:13.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.992+0000 7f8798c00640 1 -- 172.21.15.26:0/3757601831 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) ==== 80+0+0 (secure 0 0 0) 0x7f878c07aee0 con 0x7f879c06e700 2024-04-26T21:10:13.994 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-04-26T21:10:13.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.995+0000 7f87a2000640 1 -- 172.21.15.26:0/3757601831 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f877c06c8f0 msgr2=0x7f877c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:13.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.995+0000 7f87a2000640 1 --2- 172.21.15.26:0/3757601831 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f877c06c8f0 0x7f877c06edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f879c162f30 tx=0x7f8788008040 comp rx=0 tx=0).stop 2024-04-26T21:10:13.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.995+0000 7f87a2000640 1 -- 172.21.15.26:0/3757601831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f879c06e700 msgr2=0x7f879c0afec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:13.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.995+0000 7f87a2000640 1 --2- 172.21.15.26:0/3757601831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f879c06e700 0x7f879c0afec0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f878c0097a0 tx=0x7f878c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.995+0000 7f87a2000640 1 -- 172.21.15.26:0/3757601831 shutdown_connections 2024-04-26T21:10:13.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.995+0000 7f87a2000640 1 --2- 172.21.15.26:0/3757601831 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f877c06c8f0 0x7f877c06edb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.995+0000 7f87a2000640 1 --2- 172.21.15.26:0/3757601831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f879c1677a0 0x7f879c1f93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.995+0000 7f87a2000640 1 --2- 172.21.15.26:0/3757601831 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f879c15dcb0 0x7f879c15b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.995+0000 7f87a2000640 1 --2- 172.21.15.26:0/3757601831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f879c06e700 0x7f879c0afec0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:13.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.995+0000 7f87a2000640 1 -- 172.21.15.26:0/3757601831 >> 172.21.15.26:0/3757601831 conn(0x7f879c06c690 msgr2=0x7f879c06ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:13.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.995+0000 7f87a2000640 1 -- 172.21.15.26:0/3757601831 shutdown_connections 2024-04-26T21:10:13.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:13.995+0000 7f87a2000640 1 -- 172.21.15.26:0/3757601831 wait complete. 2024-04-26T21:10:14.018 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:10:14.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:10:14.018 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:10:14.172 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-04-26T21:10:14.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:10:14.173 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:10:14.218 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:10:14.219 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:10:14.219 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:10:14.260 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:10:14.260 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:10:14.260 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:10:14.260 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:10:14.301 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:10:14.301 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:10:14.301 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:10:14.301 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:10:14.350 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:10:14.350 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:10:14.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.560+0000 7f332da00640 1 Processor -- start 2024-04-26T21:10:14.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.560+0000 7f332da00640 1 -- start start 2024-04-26T21:10:14.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.562+0000 7f332da00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f33280d06f0 0x7f33280d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:14.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.562+0000 7f332da00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f33280d1670 0x7f3328071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:14.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.562+0000 7f332da00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33280cf690 con 0x7f3328145c20 2024-04-26T21:10:14.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.562+0000 7f332da00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f33280ced10 con 0x7f33280d1670 2024-04-26T21:10:14.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.562+0000 7f332da00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f33280d1ac0 con 0x7f33280d06f0 2024-04-26T21:10:14.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.562+0000 7f332d000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f33280d1670 0x7f3328071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:14.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.562+0000 7f3327e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3328145c20 0x7f3328146450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33906/0 (socket says 172.21.15.26:33906) 2024-04-26T21:10:14.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.562+0000 7f3327e00640 1 -- 172.21.15.26:0/364677835 learned_addr learned my addr 172.21.15.26:0/364677835 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:14.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.563+0000 7f3327400640 1 --2- 172.21.15.26:0/364677835 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f33280d06f0 0x7f33280d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:14.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.563+0000 7f332d000640 1 -- 172.21.15.26:0/364677835 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f33280d06f0 msgr2=0x7f33280d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:14.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.563+0000 7f332d000640 1 --2- 172.21.15.26:0/364677835 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f33280d06f0 0x7f33280d1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:14.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.563+0000 7f332d000640 1 -- 172.21.15.26:0/364677835 >> v1:172.21.15.26:6789/0 conn(0x7f3328145c20 legacy=0x7f3328146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:14.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.563+0000 7f332d000640 1 -- 172.21.15.26:0/364677835 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33280cf930 con 0x7f33280d1670 2024-04-26T21:10:14.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.564+0000 7f332d000640 1 --2- 172.21.15.26:0/364677835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33280d1670 0x7f3328071fe0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3320009d90 tx=0x7f332002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74539223e3f269f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:14.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.564+0000 7f3326a00640 1 -- 172.21.15.26:0/364677835 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3320007740 con 0x7f33280d1670 2024-04-26T21:10:14.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.564+0000 7f3326a00640 1 -- 172.21.15.26:0/364677835 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33200078e0 con 0x7f33280d1670 2024-04-26T21:10:14.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.564+0000 7f3326a00640 1 -- 172.21.15.26:0/364677835 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3320007cb0 con 0x7f33280d1670 2024-04-26T21:10:14.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.565+0000 7f332da00640 1 -- 172.21.15.26:0/364677835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33280d1670 msgr2=0x7f3328071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:14.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.565+0000 7f332da00640 1 --2- 172.21.15.26:0/364677835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33280d1670 0x7f3328071fe0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3320009d90 tx=0x7f332002a660 comp rx=0 tx=0).stop 2024-04-26T21:10:14.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.565+0000 7f332da00640 1 -- 172.21.15.26:0/364677835 shutdown_connections 2024-04-26T21:10:14.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.565+0000 7f332da00640 1 --2- 172.21.15.26:0/364677835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33280d1670 0x7f3328071fe0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:14.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.565+0000 7f332da00640 1 --2- 172.21.15.26:0/364677835 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f33280d06f0 0x7f33280d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:14.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.565+0000 7f332da00640 1 -- 172.21.15.26:0/364677835 >> 172.21.15.26:0/364677835 conn(0x7f332806c690 msgr2=0x7f332806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:14.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.565+0000 7f332da00640 1 -- 172.21.15.26:0/364677835 shutdown_connections 2024-04-26T21:10:14.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.565+0000 7f332da00640 1 -- 172.21.15.26:0/364677835 wait complete. 2024-04-26T21:10:14.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.567+0000 7f332da00640 1 Processor -- start 2024-04-26T21:10:14.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.567+0000 7f332da00640 1 -- start start 2024-04-26T21:10:14.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.568+0000 7f332da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33280d06f0 0x7f3328141950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:14.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.568+0000 7f332da00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33280d1670 0x7f33281590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:14.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.568+0000 7f332da00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3328145c20 0x7f3328155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:14.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.568+0000 7f332da00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f33280cfef0 con 0x7f33280d1670 2024-04-26T21:10:14.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.568+0000 7f332da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33280d0b50 con 0x7f33280d06f0 2024-04-26T21:10:14.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.568+0000 7f332da00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f33280d01d0 con 0x7f3328145c20 2024-04-26T21:10:14.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.568+0000 7f3327400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33280d1670 0x7f33281590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:14.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.568+0000 7f3327400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33280d1670 0x7f33281590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47026/0 (socket says 172.21.15.26:47026) 2024-04-26T21:10:14.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.568+0000 7f3327400640 1 -- 172.21.15.26:0/932196150 learned_addr learned my addr 172.21.15.26:0/932196150 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:14.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.569+0000 7f332d000640 1 --2- 172.21.15.26:0/932196150 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3328145c20 0x7f3328155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:14.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.569+0000 7f3327400640 1 -- 172.21.15.26:0/932196150 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3328145c20 msgr2=0x7f3328155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:14.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.569+0000 7f3327400640 1 --2- 172.21.15.26:0/932196150 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3328145c20 0x7f3328155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:14.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.569+0000 7f3327e00640 1 --2- 172.21.15.26:0/932196150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33280d06f0 0x7f3328141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:14.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.569+0000 7f3327400640 1 -- 172.21.15.26:0/932196150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33280d06f0 msgr2=0x7f3328141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:14.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.569+0000 7f3327400640 1 --2- 172.21.15.26:0/932196150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33280d06f0 0x7f3328141950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:14.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.569+0000 7f3327400640 1 -- 172.21.15.26:0/932196150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33281f2be0 con 0x7f33280d1670 2024-04-26T21:10:14.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.569+0000 7f3327400640 1 --2- 172.21.15.26:0/932196150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33280d1670 0x7f33281590a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f331400d800 tx=0x7f331400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:14.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.569+0000 7f331fe00640 1 -- 172.21.15.26:0/932196150 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331400b6c0 con 0x7f33280d1670 2024-04-26T21:10:14.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.570+0000 7f331fe00640 1 -- 172.21.15.26:0/932196150 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3314019070 con 0x7f33280d1670 2024-04-26T21:10:14.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.570+0000 7f331fe00640 1 -- 172.21.15.26:0/932196150 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331400ba80 con 0x7f33280d1670 2024-04-26T21:10:14.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.570+0000 7f332da00640 1 -- 172.21.15.26:0/932196150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33281596a0 con 0x7f33280d1670 2024-04-26T21:10:14.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.570+0000 7f332da00640 1 -- 172.21.15.26:0/932196150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33281f3130 con 0x7f33280d1670 2024-04-26T21:10:14.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.571+0000 7f332da00640 1 -- 172.21.15.26:0/932196150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f332806a700 con 0x7f33280d1670 2024-04-26T21:10:14.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.572+0000 7f331fe00640 1 -- 172.21.15.26:0/932196150 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f331400bc20 con 0x7f33280d1670 2024-04-26T21:10:14.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.573+0000 7f331fe00640 1 --2- 172.21.15.26:0/932196150 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f330806c8f0 0x7f330806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:14.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.573+0000 7f331fe00640 1 -- 172.21.15.26:0/932196150 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f331408c1e0 con 0x7f33280d1670 2024-04-26T21:10:14.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.573+0000 7f3327e00640 1 --2- 172.21.15.26:0/932196150 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f330806c8f0 0x7f330806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:14.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.574+0000 7f3327e00640 1 --2- 172.21.15.26:0/932196150 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f330806c8f0 0x7f330806edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3318000f60 tx=0x7f3318009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:14.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.577+0000 7f331fe00640 1 -- 172.21.15.26:0/932196150 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f331401c360 con 0x7f33280d1670 2024-04-26T21:10:14.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.821+0000 7f332da00640 1 -- 172.21.15.26:0/932196150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33280b36c0 con 0x7f33280d1670 2024-04-26T21:10:14.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.822+0000 7f331fe00640 1 -- 172.21.15.26:0/932196150 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5180 (secure 0 0 0) 0x7f331405a470 con 0x7f33280d1670 2024-04-26T21:10:14.824 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:14.824 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2349160700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2349160700},{"type":"v1","addr":"172.21.15.26:6837","nonce":2349160700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4598,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2824191762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2824191762},{"type":"v1","addr":"172.21.15.86:6834","nonce":2824191762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/263746310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":263746310},{"type":"v1","addr":"172.21.15.86:6835","nonce":263746310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:09:48.622025+0000","modified":"2024-04-26T21:10:05.296141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5058":{"gid":5058,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6836/2528180854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2528180854},{"type":"v1","addr":"172.21.15.26:6836","nonce":2528180854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5058,"qdb_cluster":[5058]},"id":2}]} 2024-04-26T21:10:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 18 2024-04-26T21:10:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.825+0000 7f332da00640 1 -- 172.21.15.26:0/932196150 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f330806c8f0 msgr2=0x7f330806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.825+0000 7f332da00640 1 --2- 172.21.15.26:0/932196150 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f330806c8f0 0x7f330806edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3318000f60 tx=0x7f3318009040 comp rx=0 tx=0).stop 2024-04-26T21:10:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.825+0000 7f332da00640 1 -- 172.21.15.26:0/932196150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33280d1670 msgr2=0x7f33281590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.825+0000 7f332da00640 1 --2- 172.21.15.26:0/932196150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33280d1670 0x7f33281590a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f331400d800 tx=0x7f331400dcc0 comp rx=0 tx=0).stop 2024-04-26T21:10:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.825+0000 7f332da00640 1 -- 172.21.15.26:0/932196150 shutdown_connections 2024-04-26T21:10:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.825+0000 7f332da00640 1 --2- 172.21.15.26:0/932196150 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f330806c8f0 0x7f330806edb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.825+0000 7f332da00640 1 --2- 172.21.15.26:0/932196150 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3328145c20 0x7f3328155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.825+0000 7f332da00640 1 --2- 172.21.15.26:0/932196150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33280d1670 0x7f33281590a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.825+0000 7f332da00640 1 --2- 172.21.15.26:0/932196150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33280d06f0 0x7f3328141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.825+0000 7f332da00640 1 -- 172.21.15.26:0/932196150 >> 172.21.15.26:0/932196150 conn(0x7f332806c690 msgr2=0x7f332815f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.826+0000 7f332da00640 1 -- 172.21.15.26:0/932196150 shutdown_connections 2024-04-26T21:10:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:14.826+0000 7f332da00640 1 -- 172.21.15.26:0/932196150 wait complete. 2024-04-26T21:10:14.845 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:10:15.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.050+0000 7f815de00640 1 Processor -- start 2024-04-26T21:10:15.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.050+0000 7f815de00640 1 -- start start 2024-04-26T21:10:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.052+0000 7f815de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8158145c20 0x7f81580d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.052+0000 7f815de00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8158146450 0x7f81580d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.052+0000 7f815de00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f81581468b0 0x7f81580d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.052+0000 7f815de00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f81580ced10 con 0x7f81581468b0 2024-04-26T21:10:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.052+0000 7f815de00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f81580cf690 con 0x7f8158145c20 2024-04-26T21:10:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.052+0000 7f815de00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f81580cf930 con 0x7f8158146450 2024-04-26T21:10:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.053+0000 7f815d400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f81581468b0 0x7f81580d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.053+0000 7f815d400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f81581468b0 0x7f81580d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47042/0 (socket says 172.21.15.26:47042) 2024-04-26T21:10:15.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.053+0000 7f815d400640 1 -- 172.21.15.26:0/504236590 learned_addr learned my addr 172.21.15.26:0/504236590 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:15.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.053+0000 7f815ca00640 1 --2- 172.21.15.26:0/504236590 >> v2:172.21.15.86:3300/0 conn(0x7f8158145c20 0x7f81580d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.053+0000 7f8157e00640 1 --2- 172.21.15.26:0/504236590 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8158146450 0x7f81580d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.053+0000 7f815d400640 1 -- 172.21.15.26:0/504236590 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8158146450 msgr2=0x7f81580d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.053+0000 7f815d400640 1 --2- 172.21.15.26:0/504236590 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8158146450 0x7f81580d1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.053+0000 7f815d400640 1 -- 172.21.15.26:0/504236590 >> v2:172.21.15.86:3300/0 conn(0x7f8158145c20 msgr2=0x7f81580d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.053+0000 7f815d400640 1 --2- 172.21.15.26:0/504236590 >> v2:172.21.15.86:3300/0 conn(0x7f8158145c20 0x7f81580d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.053+0000 7f815d400640 1 -- 172.21.15.26:0/504236590 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81580cfc10 con 0x7f81581468b0 2024-04-26T21:10:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.053+0000 7f8157e00640 1 --2- 172.21.15.26:0/504236590 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8158146450 0x7f81580d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.054+0000 7f815d400640 1 --2- 172.21.15.26:0/504236590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81581468b0 0x7f81580d1670 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f81580d1cc0 tx=0x7f815002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d023afd66d514c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:15.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.054+0000 7f8157400640 1 -- 172.21.15.26:0/504236590 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150036070 con 0x7f81581468b0 2024-04-26T21:10:15.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.054+0000 7f8157400640 1 -- 172.21.15.26:0/504236590 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8150031600 con 0x7f81581468b0 2024-04-26T21:10:15.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.054+0000 7f8157400640 1 -- 172.21.15.26:0/504236590 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81500319d0 con 0x7f81581468b0 2024-04-26T21:10:15.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.055+0000 7f815de00640 1 -- 172.21.15.26:0/504236590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81581468b0 msgr2=0x7f81580d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:15.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.055+0000 7f815de00640 1 --2- 172.21.15.26:0/504236590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81581468b0 0x7f81580d1670 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f81580d1cc0 tx=0x7f815002a6a0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.055+0000 7f815de00640 1 -- 172.21.15.26:0/504236590 shutdown_connections 2024-04-26T21:10:15.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.055+0000 7f815de00640 1 --2- 172.21.15.26:0/504236590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81581468b0 0x7f81580d1670 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.055+0000 7f815de00640 1 --2- 172.21.15.26:0/504236590 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8158146450 0x7f81580d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.055+0000 7f815de00640 1 --2- 172.21.15.26:0/504236590 >> v2:172.21.15.86:3300/0 conn(0x7f8158145c20 0x7f81580d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.055+0000 7f815de00640 1 -- 172.21.15.26:0/504236590 >> 172.21.15.26:0/504236590 conn(0x7f815806c690 msgr2=0x7f815806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:15.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.055+0000 7f815de00640 1 -- 172.21.15.26:0/504236590 shutdown_connections 2024-04-26T21:10:15.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.055+0000 7f815de00640 1 -- 172.21.15.26:0/504236590 wait complete. 2024-04-26T21:10:15.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.057+0000 7f815de00640 1 Processor -- start 2024-04-26T21:10:15.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.057+0000 7f815de00640 1 -- start start 2024-04-26T21:10:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.058+0000 7f815de00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8158145c20 0x7f8158141770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.058+0000 7f815de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8158146450 0x7f815815c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.058+0000 7f815de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81581468b0 0x7f815815d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.058+0000 7f815de00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8158146000 con 0x7f8158146450 2024-04-26T21:10:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.058+0000 7f815de00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f81580d01d0 con 0x7f81581468b0 2024-04-26T21:10:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.058+0000 7f815de00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8158158340 con 0x7f8158145c20 2024-04-26T21:10:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.058+0000 7f8157e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8158146450 0x7f815815c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.058+0000 7f8157e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8158146450 0x7f815815c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47050/0 (socket says 172.21.15.26:47050) 2024-04-26T21:10:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.058+0000 7f8157e00640 1 -- 172.21.15.26:0/2730414345 learned_addr learned my addr 172.21.15.26:0/2730414345 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:15.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.059+0000 7f8157e00640 1 -- 172.21.15.26:0/2730414345 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8158145c20 msgr2=0x7f8158141770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:15.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.059+0000 7f815ca00640 1 --2- 172.21.15.26:0/2730414345 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8158145c20 0x7f8158141770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.059+0000 7f815d400640 1 --2- 172.21.15.26:0/2730414345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81581468b0 0x7f815815d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.059+0000 7f8157e00640 1 --2- 172.21.15.26:0/2730414345 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8158145c20 0x7f8158141770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.059+0000 7f8157e00640 1 -- 172.21.15.26:0/2730414345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81581468b0 msgr2=0x7f815815d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.059+0000 7f8157e00640 1 --2- 172.21.15.26:0/2730414345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81581468b0 0x7f815815d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.059+0000 7f8157e00640 1 -- 172.21.15.26:0/2730414345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8158141cb0 con 0x7f8158146450 2024-04-26T21:10:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.059+0000 7f8157e00640 1 --2- 172.21.15.26:0/2730414345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8158146450 0x7f815815c230 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f814400c820 tx=0x7f814400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.059+0000 7f8154c00640 1 -- 172.21.15.26:0/2730414345 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8144015070 con 0x7f8158146450 2024-04-26T21:10:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.059+0000 7f8154c00640 1 -- 172.21.15.26:0/2730414345 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8144007a30 con 0x7f8158146450 2024-04-26T21:10:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.059+0000 7f815de00640 1 -- 172.21.15.26:0/2730414345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f815815ca60 con 0x7f8158146450 2024-04-26T21:10:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.060+0000 7f8154c00640 1 -- 172.21.15.26:0/2730414345 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8144007e00 con 0x7f8158146450 2024-04-26T21:10:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.060+0000 7f815de00640 1 -- 172.21.15.26:0/2730414345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f815815c770 con 0x7f8158146450 2024-04-26T21:10:15.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.061+0000 7f815de00640 1 -- 172.21.15.26:0/2730414345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f815806a700 con 0x7f8158146450 2024-04-26T21:10:15.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.062+0000 7f8154c00640 1 -- 172.21.15.26:0/2730414345 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8144018070 con 0x7f8158146450 2024-04-26T21:10:15.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.063+0000 7f8154c00640 1 --2- 172.21.15.26:0/2730414345 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f813806c8f0 0x7f813806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.063+0000 7f8154c00640 1 -- 172.21.15.26:0/2730414345 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f814408bcc0 con 0x7f8158146450 2024-04-26T21:10:15.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.063+0000 7f815ca00640 1 --2- 172.21.15.26:0/2730414345 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f813806c8f0 0x7f813806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.064+0000 7f815ca00640 1 --2- 172.21.15.26:0/2730414345 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f813806c8f0 0x7f813806edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f814c006fd0 tx=0x7f814c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:15.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.067+0000 7f8154c00640 1 -- 172.21.15.26:0/2730414345 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8144092080 con 0x7f8158146450 2024-04-26T21:10:15.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.273+0000 7f815de00640 1 -- 172.21.15.26:0/2730414345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f815815ae90 con 0x7f8158146450 2024-04-26T21:10:15.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.277+0000 7f8154c00640 1 -- 172.21.15.26:0/2730414345 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+18433 (secure 0 0 0) 0x7f8144059f50 con 0x7f8158146450 2024-04-26T21:10:15.279 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:15.279 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":37,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:10:06.822858+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:46.646256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:47.613883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-04-26T21:10:04.249730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:10:15.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.279+0000 7f815de00640 1 -- 172.21.15.26:0/2730414345 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f813806c8f0 msgr2=0x7f813806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:15.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.279+0000 7f815de00640 1 --2- 172.21.15.26:0/2730414345 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f813806c8f0 0x7f813806edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f814c006fd0 tx=0x7f814c008040 comp rx=0 tx=0).stop 2024-04-26T21:10:15.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.279+0000 7f815de00640 1 -- 172.21.15.26:0/2730414345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8158146450 msgr2=0x7f815815c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:15.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.279+0000 7f815de00640 1 --2- 172.21.15.26:0/2730414345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8158146450 0x7f815815c230 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f814400c820 tx=0x7f814400cc20 comp rx=0 tx=0).stop 2024-04-26T21:10:15.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.279+0000 7f815de00640 1 -- 172.21.15.26:0/2730414345 shutdown_connections 2024-04-26T21:10:15.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.279+0000 7f815de00640 1 --2- 172.21.15.26:0/2730414345 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f813806c8f0 0x7f813806edb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.279+0000 7f815de00640 1 --2- 172.21.15.26:0/2730414345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81581468b0 0x7f815815d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.279+0000 7f815de00640 1 --2- 172.21.15.26:0/2730414345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8158146450 0x7f815815c230 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.279+0000 7f815de00640 1 --2- 172.21.15.26:0/2730414345 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8158145c20 0x7f8158141770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.279+0000 7f815de00640 1 -- 172.21.15.26:0/2730414345 >> 172.21.15.26:0/2730414345 conn(0x7f815806c690 msgr2=0x7f8158160d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:15.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.279+0000 7f815de00640 1 -- 172.21.15.26:0/2730414345 shutdown_connections 2024-04-26T21:10:15.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.279+0000 7f815de00640 1 -- 172.21.15.26:0/2730414345 wait complete. 2024-04-26T21:10:15.301 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:08:31.322511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:09:46.646256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:09:47.613883+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'data_new', 'create_time': '2024-04-26T21:10:04.249730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:10:15.301 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:10:15.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.513+0000 7fcdce600640 1 Processor -- start 2024-04-26T21:10:15.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.513+0000 7fcdce600640 1 -- start start 2024-04-26T21:10:15.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.515+0000 7fcdce600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdc8160a80 0x7fcdc80b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.515+0000 7fcdce600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcdc8160e60 0x7fcdc806e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.515+0000 7fcdce600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdc8072d90 con 0x7fcdc8072490 2024-04-26T21:10:15.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.515+0000 7fcdce600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fcdc80b5be0 con 0x7fcdc8160e60 2024-04-26T21:10:15.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.515+0000 7fcdce600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcdc8073070 con 0x7fcdc8160a80 2024-04-26T21:10:15.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.515+0000 7fcdc7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcdc8160e60 0x7fcdc806e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.515+0000 7fcdc7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcdc8160e60 0x7fcdc806e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47064/0 (socket says 172.21.15.26:47064) 2024-04-26T21:10:15.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.515+0000 7fcdc7e00640 1 -- 172.21.15.26:0/1778466394 learned_addr learned my addr 172.21.15.26:0/1778466394 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:15.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.515+0000 7fcdcd200640 1 --2- 172.21.15.26:0/1778466394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdc8160a80 0x7fcdc80b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.516+0000 7fcdc7e00640 1 -- 172.21.15.26:0/1778466394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdc8160a80 msgr2=0x7fcdc80b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:15.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.516+0000 7fcdc7e00640 1 --2- 172.21.15.26:0/1778466394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdc8160a80 0x7fcdc80b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.516+0000 7fcdc7e00640 1 -- 172.21.15.26:0/1778466394 >> v1:172.21.15.86:6789/0 conn(0x7fcdc8072490 legacy=0x7fcdc80b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:15.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.516+0000 7fcdc7e00640 1 -- 172.21.15.26:0/1778466394 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc8073350 con 0x7fcdc8160e60 2024-04-26T21:10:15.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.517+0000 7fcdc7e00640 1 --2- 172.21.15.26:0/1778466394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdc8160e60 0x7fcdc806e210 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4009870 tx=0x7fcdb402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efdd01536d93619 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:15.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.517+0000 7fcdc7400640 1 -- 172.21.15.26:0/1778466394 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4037070 con 0x7fcdc8160e60 2024-04-26T21:10:15.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.517+0000 7fcdc7400640 1 -- 172.21.15.26:0/1778466394 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdb4032440 con 0x7fcdc8160e60 2024-04-26T21:10:15.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.517+0000 7fcdc7400640 1 -- 172.21.15.26:0/1778466394 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4032810 con 0x7fcdc8160e60 2024-04-26T21:10:15.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.518+0000 7fcdce600640 1 -- 172.21.15.26:0/1778466394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdc8160e60 msgr2=0x7fcdc806e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:15.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.518+0000 7fcdce600640 1 --2- 172.21.15.26:0/1778466394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdc8160e60 0x7fcdc806e210 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4009870 tx=0x7fcdb402a210 comp rx=0 tx=0).stop 2024-04-26T21:10:15.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.518+0000 7fcdce600640 1 -- 172.21.15.26:0/1778466394 shutdown_connections 2024-04-26T21:10:15.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.518+0000 7fcdce600640 1 --2- 172.21.15.26:0/1778466394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdc8160e60 0x7fcdc806e210 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.518+0000 7fcdce600640 1 --2- 172.21.15.26:0/1778466394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdc8160a80 0x7fcdc80b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.518+0000 7fcdce600640 1 -- 172.21.15.26:0/1778466394 >> 172.21.15.26:0/1778466394 conn(0x7fcdc80b2a10 msgr2=0x7fcdc80b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:15.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.518+0000 7fcdce600640 1 -- 172.21.15.26:0/1778466394 shutdown_connections 2024-04-26T21:10:15.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.518+0000 7fcdce600640 1 -- 172.21.15.26:0/1778466394 wait complete. 2024-04-26T21:10:15.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.520+0000 7fcdce600640 1 Processor -- start 2024-04-26T21:10:15.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.520+0000 7fcdce600640 1 -- start start 2024-04-26T21:10:15.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.521+0000 7fcdce600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdc8072490 0x7fcdc80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.521+0000 7fcdce600640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdc8160a80 0x7fcdc81501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.521+0000 7fcdce600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdc8160e60 0x7fcdc81589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.521+0000 7fcdce600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc81612a0 con 0x7fcdc8160e60 2024-04-26T21:10:15.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.521+0000 7fcdce600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc80b5820 con 0x7fcdc8072490 2024-04-26T21:10:15.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.521+0000 7fcdce600640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcdc815b250 con 0x7fcdc8160a80 2024-04-26T21:10:15.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.521+0000 7fcdcdc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdc8160e60 0x7fcdc81589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.521+0000 7fcdcdc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdc8160e60 0x7fcdc81589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47076/0 (socket says 172.21.15.26:47076) 2024-04-26T21:10:15.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.521+0000 7fcdcdc00640 1 -- 172.21.15.26:0/2828942658 learned_addr learned my addr 172.21.15.26:0/2828942658 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:15.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.521+0000 7fcdcd200640 1 --2- 172.21.15.26:0/2828942658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdc8072490 0x7fcdc80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.521+0000 7fcdc7e00640 1 --2- 172.21.15.26:0/2828942658 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdc8160a80 0x7fcdc81501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.522+0000 7fcdcdc00640 1 -- 172.21.15.26:0/2828942658 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdc8160a80 msgr2=0x7fcdc81501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:15.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.522+0000 7fcdcdc00640 1 --2- 172.21.15.26:0/2828942658 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdc8160a80 0x7fcdc81501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.522+0000 7fcdcdc00640 1 -- 172.21.15.26:0/2828942658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdc8072490 msgr2=0x7fcdc80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:15.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.522+0000 7fcdcdc00640 1 --2- 172.21.15.26:0/2828942658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdc8072490 0x7fcdc80cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.522+0000 7fcdcdc00640 1 -- 172.21.15.26:0/2828942658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc8151b70 con 0x7fcdc8160e60 2024-04-26T21:10:15.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.522+0000 7fcdc7e00640 1 --2- 172.21.15.26:0/2828942658 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdc8160a80 0x7fcdc81501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:15.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.522+0000 7fcdcdc00640 1 --2- 172.21.15.26:0/2828942658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdc8160e60 0x7fcdc81589c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000c410 tx=0x7fcdc000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:15.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.522+0000 7fcdc4c00640 1 -- 172.21.15.26:0/2828942658 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0011790 con 0x7fcdc8160e60 2024-04-26T21:10:15.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.522+0000 7fcdc4c00640 1 -- 172.21.15.26:0/2828942658 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdc0017070 con 0x7fcdc8160e60 2024-04-26T21:10:15.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.522+0000 7fcdc4c00640 1 -- 172.21.15.26:0/2828942658 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0011bb0 con 0x7fcdc8160e60 2024-04-26T21:10:15.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.522+0000 7fcdce600640 1 -- 172.21.15.26:0/2828942658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc8146b40 con 0x7fcdc8160e60 2024-04-26T21:10:15.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.522+0000 7fcdce600640 1 -- 172.21.15.26:0/2828942658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc81520f0 con 0x7fcdc8160e60 2024-04-26T21:10:15.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.523+0000 7fcdce600640 1 -- 172.21.15.26:0/2828942658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdc806a700 con 0x7fcdc8160e60 2024-04-26T21:10:15.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.524+0000 7fcdc4c00640 1 -- 172.21.15.26:0/2828942658 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcdc001c460 con 0x7fcdc8160e60 2024-04-26T21:10:15.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.525+0000 7fcdc4c00640 1 --2- 172.21.15.26:0/2828942658 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcda806c920 0x7fcda806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.525+0000 7fcdc4c00640 1 -- 172.21.15.26:0/2828942658 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7fcdc008ba80 con 0x7fcdc8160e60 2024-04-26T21:10:15.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.526+0000 7fcdcd200640 1 --2- 172.21.15.26:0/2828942658 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcda806c920 0x7fcda806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.526+0000 7fcdcd200640 1 --2- 172.21.15.26:0/2828942658 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcda806c920 0x7fcda806ede0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009f30 tx=0x7fcdb800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:15.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.529+0000 7fcdc4c00640 1 -- 172.21.15.26:0/2828942658 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcdc008de20 con 0x7fcdc8160e60 2024-04-26T21:10:15.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.732+0000 7fcdce600640 1 -- 172.21.15.26:0/2828942658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcdc8072270 con 0x7fcdc8160e60 2024-04-26T21:10:15.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.733+0000 7fcdc4c00640 1 -- 172.21.15.26:0/2828942658 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) ==== 93+0+10 (secure 0 0 0) 0x7fcdc0059d10 con 0x7fcdc8160e60 2024-04-26T21:10:15.735 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-04-26T21:10:15.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.736+0000 7fcdce600640 1 -- 172.21.15.26:0/2828942658 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcda806c920 msgr2=0x7fcda806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:15.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.736+0000 7fcdce600640 1 --2- 172.21.15.26:0/2828942658 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcda806c920 0x7fcda806ede0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009f30 tx=0x7fcdb800b040 comp rx=0 tx=0).stop 2024-04-26T21:10:15.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.736+0000 7fcdce600640 1 -- 172.21.15.26:0/2828942658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdc8160e60 msgr2=0x7fcdc81589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:15.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.736+0000 7fcdce600640 1 --2- 172.21.15.26:0/2828942658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdc8160e60 0x7fcdc81589c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000c410 tx=0x7fcdc000c8d0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.736+0000 7fcdce600640 1 -- 172.21.15.26:0/2828942658 shutdown_connections 2024-04-26T21:10:15.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.736+0000 7fcdce600640 1 --2- 172.21.15.26:0/2828942658 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcda806c920 0x7fcda806ede0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.736+0000 7fcdce600640 1 --2- 172.21.15.26:0/2828942658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdc8160e60 0x7fcdc81589c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.736+0000 7fcdce600640 1 --2- 172.21.15.26:0/2828942658 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdc8160a80 0x7fcdc81501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.736+0000 7fcdce600640 1 --2- 172.21.15.26:0/2828942658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdc8072490 0x7fcdc80cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.736+0000 7fcdce600640 1 -- 172.21.15.26:0/2828942658 >> 172.21.15.26:0/2828942658 conn(0x7fcdc80b2a10 msgr2=0x7fcdc815a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:15.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.736+0000 7fcdce600640 1 -- 172.21.15.26:0/2828942658 shutdown_connections 2024-04-26T21:10:15.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.736+0000 7fcdce600640 1 -- 172.21.15.26:0/2828942658 wait complete. 2024-04-26T21:10:15.755 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:10:15.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.971+0000 7fcdb4c00640 1 Processor -- start 2024-04-26T21:10:15.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.971+0000 7fcdb4c00640 1 -- start start 2024-04-26T21:10:15.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.973+0000 7fcdb4c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcdb014ce50 0x7fcdb00cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.973+0000 7fcdb4c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdb0072780 0x7fcdb006ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.973+0000 7fcdb4c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdb00cfe60 con 0x7fcdb00b3630 2024-04-26T21:10:15.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.973+0000 7fcdb4c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fcdb006e290 con 0x7fcdb014ce50 2024-04-26T21:10:15.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.973+0000 7fcdb4c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcdb006e570 con 0x7fcdb0072780 2024-04-26T21:10:15.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.973+0000 7fcdad400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcdb00b3630 0x7fcdb0072070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33908/0 (socket says 172.21.15.26:33908) 2024-04-26T21:10:15.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.973+0000 7fcdad400640 1 -- 172.21.15.26:0/4074771377 learned_addr learned my addr 172.21.15.26:0/4074771377 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:15.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.973+0000 7fcdade00640 1 --2- 172.21.15.26:0/4074771377 >> v2:172.21.15.26:3300/0 conn(0x7fcdb014ce50 0x7fcdb00cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.974+0000 7fcdade00640 1 -- 172.21.15.26:0/4074771377 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdb0072780 msgr2=0x7fcdb006ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:15.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.974+0000 7fcdae800640 1 --2- 172.21.15.26:0/4074771377 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdb0072780 0x7fcdb006ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.974+0000 7fcdade00640 1 --2- 172.21.15.26:0/4074771377 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdb0072780 0x7fcdb006ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.974+0000 7fcdade00640 1 -- 172.21.15.26:0/4074771377 >> v1:172.21.15.26:6789/0 conn(0x7fcdb00b3630 legacy=0x7fcdb0072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:15.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.974+0000 7fcdade00640 1 -- 172.21.15.26:0/4074771377 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb006e850 con 0x7fcdb014ce50 2024-04-26T21:10:15.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.974+0000 7fcdaca00640 1 -- 172.21.15.26:0/4074771377 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 22684778 0 0) 0x7fcdb00cfe60 con 0x7fcdb00b3630 2024-04-26T21:10:15.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.975+0000 7fcdade00640 1 --2- 172.21.15.26:0/4074771377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb014ce50 0x7fcdb00cf510 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcda00097a0 tx=0x7fcda002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e25390ac2f650f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.975+0000 7fcdaca00640 1 -- 172.21.15.26:0/4074771377 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda0037070 con 0x7fcdb014ce50 2024-04-26T21:10:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.975+0000 7fcdaca00640 1 -- 172.21.15.26:0/4074771377 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda0032440 con 0x7fcdb014ce50 2024-04-26T21:10:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.975+0000 7fcdaca00640 1 -- 172.21.15.26:0/4074771377 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda0032810 con 0x7fcdb014ce50 2024-04-26T21:10:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.976+0000 7fcdb4c00640 1 -- 172.21.15.26:0/4074771377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb014ce50 msgr2=0x7fcdb00cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.976+0000 7fcdb4c00640 1 --2- 172.21.15.26:0/4074771377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb014ce50 0x7fcdb00cf510 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcda00097a0 tx=0x7fcda002a180 comp rx=0 tx=0).stop 2024-04-26T21:10:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.976+0000 7fcdb4c00640 1 -- 172.21.15.26:0/4074771377 shutdown_connections 2024-04-26T21:10:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.976+0000 7fcdb4c00640 1 --2- 172.21.15.26:0/4074771377 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdb0072780 0x7fcdb006ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.976+0000 7fcdb4c00640 1 --2- 172.21.15.26:0/4074771377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb014ce50 0x7fcdb00cf510 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.976+0000 7fcdb4c00640 1 -- 172.21.15.26:0/4074771377 >> 172.21.15.26:0/4074771377 conn(0x7fcdb00b2a80 msgr2=0x7fcdb00b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.976+0000 7fcdb4c00640 1 -- 172.21.15.26:0/4074771377 shutdown_connections 2024-04-26T21:10:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.976+0000 7fcdb4c00640 1 -- 172.21.15.26:0/4074771377 wait complete. 2024-04-26T21:10:15.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.978+0000 7fcdb4c00640 1 Processor -- start 2024-04-26T21:10:15.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.978+0000 7fcdb4c00640 1 -- start start 2024-04-26T21:10:15.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.979+0000 7fcdb4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdb0072780 0x7fcdb0147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.979+0000 7fcdb4c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdb00b3630 0x7fcdb01eea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.979+0000 7fcdb4c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb014ce50 0x7fcdb01ef3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.979+0000 7fcdb4c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb00cfa50 con 0x7fcdb014ce50 2024-04-26T21:10:15.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.979+0000 7fcdb4c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb00d0140 con 0x7fcdb0072780 2024-04-26T21:10:15.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.979+0000 7fcdb4c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fcdb00cfbd0 con 0x7fcdb00b3630 2024-04-26T21:10:15.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.979+0000 7fcdae800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb014ce50 0x7fcdb01ef3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.979+0000 7fcdae800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb014ce50 0x7fcdb01ef3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47094/0 (socket says 172.21.15.26:47094) 2024-04-26T21:10:15.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.979+0000 7fcdae800640 1 -- 172.21.15.26:0/3983806312 learned_addr learned my addr 172.21.15.26:0/3983806312 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.980+0000 7fcdad400640 1 --2- 172.21.15.26:0/3983806312 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdb00b3630 0x7fcdb01eea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.980+0000 7fcdade00640 1 --2- 172.21.15.26:0/3983806312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdb0072780 0x7fcdb0147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.980+0000 7fcdae800640 1 -- 172.21.15.26:0/3983806312 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdb00b3630 msgr2=0x7fcdb01eea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.980+0000 7fcdae800640 1 --2- 172.21.15.26:0/3983806312 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdb00b3630 0x7fcdb01eea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.980+0000 7fcdae800640 1 -- 172.21.15.26:0/3983806312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdb0072780 msgr2=0x7fcdb0147ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.980+0000 7fcdae800640 1 --2- 172.21.15.26:0/3983806312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdb0072780 0x7fcdb0147ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.980+0000 7fcdae800640 1 -- 172.21.15.26:0/3983806312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb01edb80 con 0x7fcdb014ce50 2024-04-26T21:10:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.980+0000 7fcdae800640 1 --2- 172.21.15.26:0/3983806312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb014ce50 0x7fcdb01ef3e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcda800cd40 tx=0x7fcda800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.981+0000 7fcda6000640 1 -- 172.21.15.26:0/3983806312 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8003b90 con 0x7fcdb014ce50 2024-04-26T21:10:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.981+0000 7fcda6000640 1 -- 172.21.15.26:0/3983806312 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda8003d30 con 0x7fcdb014ce50 2024-04-26T21:10:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.981+0000 7fcda6000640 1 -- 172.21.15.26:0/3983806312 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8004240 con 0x7fcdb014ce50 2024-04-26T21:10:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.981+0000 7fcdb4c00640 1 -- 172.21.15.26:0/3983806312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb01ede60 con 0x7fcdb014ce50 2024-04-26T21:10:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.981+0000 7fcdb4c00640 1 -- 172.21.15.26:0/3983806312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb01485b0 con 0x7fcdb014ce50 2024-04-26T21:10:15.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.982+0000 7fcdb4c00640 1 -- 172.21.15.26:0/3983806312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdb01ee380 con 0x7fcdb014ce50 2024-04-26T21:10:15.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.983+0000 7fcda6000640 1 -- 172.21.15.26:0/3983806312 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcda80043e0 con 0x7fcdb014ce50 2024-04-26T21:10:15.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.984+0000 7fcda6000640 1 --2- 172.21.15.26:0/3983806312 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcd9006c8f0 0x7fcd9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:15.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.985+0000 7fcda6000640 1 -- 172.21.15.26:0/3983806312 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7fcda808c7a0 con 0x7fcdb014ce50 2024-04-26T21:10:15.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.985+0000 7fcdade00640 1 --2- 172.21.15.26:0/3983806312 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcd9006c8f0 0x7fcd9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:15.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.985+0000 7fcdade00640 1 --2- 172.21.15.26:0/3983806312 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcd9006c8f0 0x7fcd9006edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcda0034000 tx=0x7fcda002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:15.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:15.989+0000 7fcda6000640 1 -- 172.21.15.26:0/3983806312 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcdb01ee380 con 0x7fcdb014ce50 2024-04-26T21:10:16.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.191+0000 7fcdb4c00640 1 -- 172.21.15.26:0/3983806312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcdb006a700 con 0x7fcdb014ce50 2024-04-26T21:10:16.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.192+0000 7fcda6000640 1 -- 172.21.15.26:0/3983806312 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) ==== 104+0+11 (secure 0 0 0) 0x7fcda801d340 con 0x7fcdb014ce50 2024-04-26T21:10:16.194 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-26T21:10:16.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.195+0000 7fcdb4c00640 1 -- 172.21.15.26:0/3983806312 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcd9006c8f0 msgr2=0x7fcd9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:16.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.195+0000 7fcdb4c00640 1 --2- 172.21.15.26:0/3983806312 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcd9006c8f0 0x7fcd9006edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcda0034000 tx=0x7fcda002e000 comp rx=0 tx=0).stop 2024-04-26T21:10:16.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.195+0000 7fcdb4c00640 1 -- 172.21.15.26:0/3983806312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb014ce50 msgr2=0x7fcdb01ef3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:16.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.195+0000 7fcdb4c00640 1 --2- 172.21.15.26:0/3983806312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb014ce50 0x7fcdb01ef3e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcda800cd40 tx=0x7fcda800a660 comp rx=0 tx=0).stop 2024-04-26T21:10:16.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.195+0000 7fcdb4c00640 1 -- 172.21.15.26:0/3983806312 shutdown_connections 2024-04-26T21:10:16.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.195+0000 7fcdb4c00640 1 --2- 172.21.15.26:0/3983806312 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fcd9006c8f0 0x7fcd9006edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:16.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.195+0000 7fcdb4c00640 1 --2- 172.21.15.26:0/3983806312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb014ce50 0x7fcdb01ef3e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:16.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.195+0000 7fcdb4c00640 1 --2- 172.21.15.26:0/3983806312 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdb00b3630 0x7fcdb01eea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:16.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.195+0000 7fcdb4c00640 1 --2- 172.21.15.26:0/3983806312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdb0072780 0x7fcdb0147ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:16.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.195+0000 7fcdb4c00640 1 -- 172.21.15.26:0/3983806312 >> 172.21.15.26:0/3983806312 conn(0x7fcdb00b2a80 msgr2=0x7fcdb00b65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:16.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.195+0000 7fcdb4c00640 1 -- 172.21.15.26:0/3983806312 shutdown_connections 2024-04-26T21:10:16.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.195+0000 7fcdb4c00640 1 -- 172.21.15.26:0/3983806312 wait complete. 2024-04-26T21:10:16.219 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:10:16.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.430+0000 7f872e200640 1 Processor -- start 2024-04-26T21:10:16.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.431+0000 7f872e200640 1 -- start start 2024-04-26T21:10:16.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.432+0000 7f872e200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f872806f9f0 0x7f8728160220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:16.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.432+0000 7f872e200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f87280734f0 0x7f8728073940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:16.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.432+0000 7f872e200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8728163130 con 0x7f8728145790 2024-04-26T21:10:16.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.432+0000 7f872e200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f87280730c0 con 0x7f87280734f0 2024-04-26T21:10:16.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.432+0000 7f872e200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8728160760 con 0x7f872806f9f0 2024-04-26T21:10:16.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.433+0000 7f8727e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f872806f9f0 0x7f8728160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:16.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.433+0000 7f8727e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f872806f9f0 0x7f8728160220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.26:51960/0 (socket says 172.21.15.26:51960) 2024-04-26T21:10:16.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.433+0000 7f8727e00640 1 -- 172.21.15.26:0/2697909100 learned_addr learned my addr 172.21.15.26:0/2697909100 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:16.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.433+0000 7f872d800640 1 --2- 172.21.15.26:0/2697909100 >> v2:172.21.15.86:3300/0 conn(0x7f87280734f0 0x7f8728073940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:16.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.434+0000 7f8727400640 1 -- 172.21.15.26:0/2697909100 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1981471953 0 0) 0x7f8728163130 con 0x7f8728145790 2024-04-26T21:10:16.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.434+0000 7f8727400640 1 -- 172.21.15.26:0/2697909100 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8710003520 con 0x7f8728145790 2024-04-26T21:10:16.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.434+0000 7f8727e00640 1 -- 172.21.15.26:0/2697909100 >> v2:172.21.15.86:3300/0 conn(0x7f87280734f0 msgr2=0x7f8728073940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:16.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.434+0000 7f8727e00640 1 --2- 172.21.15.26:0/2697909100 >> v2:172.21.15.86:3300/0 conn(0x7f87280734f0 0x7f8728073940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:16.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.434+0000 7f8727e00640 1 -- 172.21.15.26:0/2697909100 >> v1:172.21.15.86:6789/0 conn(0x7f8728145790 legacy=0x7f872806f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:16.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.434+0000 7f8727e00640 1 -- 172.21.15.26:0/2697909100 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8728163410 con 0x7f872806f9f0 2024-04-26T21:10:16.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.434+0000 7f872d800640 1 --2- 172.21.15.26:0/2697909100 >> v2:172.21.15.86:3300/0 conn(0x7f87280734f0 0x7f8728073940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:10:16.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.435+0000 7f8727e00640 1 --2- 172.21.15.26:0/2697909100 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f872806f9f0 0x7f8728160220 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8714009870 tx=0x7f871402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba495ba1631aed75 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:16.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.435+0000 7f8727400640 1 -- 172.21.15.26:0/2697909100 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714037070 con 0x7f872806f9f0 2024-04-26T21:10:16.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.435+0000 7f8727400640 1 -- 172.21.15.26:0/2697909100 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8714032440 con 0x7f872806f9f0 2024-04-26T21:10:16.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.435+0000 7f8727400640 1 -- 172.21.15.26:0/2697909100 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87140327f0 con 0x7f872806f9f0 2024-04-26T21:10:16.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.436+0000 7f872e200640 1 -- 172.21.15.26:0/2697909100 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f872806f9f0 msgr2=0x7f8728160220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:16.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.436+0000 7f872e200640 1 --2- 172.21.15.26:0/2697909100 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f872806f9f0 0x7f8728160220 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8714009870 tx=0x7f871402a1c0 comp rx=0 tx=0).stop 2024-04-26T21:10:16.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.436+0000 7f872e200640 1 -- 172.21.15.26:0/2697909100 shutdown_connections 2024-04-26T21:10:16.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.436+0000 7f872e200640 1 --2- 172.21.15.26:0/2697909100 >> v2:172.21.15.86:3300/0 conn(0x7f87280734f0 0x7f8728073940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:16.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.436+0000 7f872e200640 1 --2- 172.21.15.26:0/2697909100 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f872806f9f0 0x7f8728160220 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:16.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.436+0000 7f872e200640 1 -- 172.21.15.26:0/2697909100 >> 172.21.15.26:0/2697909100 conn(0x7f87280cd7d0 msgr2=0x7f87280cdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:16.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.436+0000 7f872e200640 1 -- 172.21.15.26:0/2697909100 shutdown_connections 2024-04-26T21:10:16.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.436+0000 7f872e200640 1 -- 172.21.15.26:0/2697909100 wait complete. 2024-04-26T21:10:16.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.438+0000 7f872e200640 1 Processor -- start 2024-04-26T21:10:16.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.438+0000 7f872e200640 1 -- start start 2024-04-26T21:10:16.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.439+0000 7f872e200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f872806f9f0 0x7f872813c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:16.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.439+0000 7f872e200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87280734f0 0x7f87280b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:16.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.439+0000 7f872e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8728145790 0x7f87280b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:16.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.439+0000 7f872e200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f87281639d0 con 0x7f872806f9f0 2024-04-26T21:10:16.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.439+0000 7f872e200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8728145b70 con 0x7f8728145790 2024-04-26T21:10:16.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.439+0000 7f872e200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f87280afca0 con 0x7f87280734f0 2024-04-26T21:10:16.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.439+0000 7f872ce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f872806f9f0 0x7f872813c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:16.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.440+0000 7f872ce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f872806f9f0 0x7f872813c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47098/0 (socket says 172.21.15.26:47098) 2024-04-26T21:10:16.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.440+0000 7f872ce00640 1 -- 172.21.15.26:0/1055694703 learned_addr learned my addr 172.21.15.26:0/1055694703 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:16.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.440+0000 7f8727e00640 1 --2- 172.21.15.26:0/1055694703 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87280734f0 0x7f87280b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:16.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.440+0000 7f872d800640 1 --2- 172.21.15.26:0/1055694703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8728145790 0x7f87280b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:16.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.440+0000 7f872ce00640 1 -- 172.21.15.26:0/1055694703 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87280734f0 msgr2=0x7f87280b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:16.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.440+0000 7f872ce00640 1 --2- 172.21.15.26:0/1055694703 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87280734f0 0x7f87280b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:16.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.440+0000 7f872ce00640 1 -- 172.21.15.26:0/1055694703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8728145790 msgr2=0x7f87280b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:16.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.440+0000 7f872ce00640 1 --2- 172.21.15.26:0/1055694703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8728145790 0x7f87280b4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:16.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.440+0000 7f872ce00640 1 -- 172.21.15.26:0/1055694703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f872813d050 con 0x7f872806f9f0 2024-04-26T21:10:16.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.440+0000 7f872d800640 1 --2- 172.21.15.26:0/1055694703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8728145790 0x7f87280b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:16.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.440+0000 7f872ce00640 1 --2- 172.21.15.26:0/1055694703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f872806f9f0 0x7f872813c670 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f871c00c390 tx=0x7f871c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:16.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.441+0000 7f8724c00640 1 -- 172.21.15.26:0/1055694703 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c017070 con 0x7f872806f9f0 2024-04-26T21:10:16.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.441+0000 7f8724c00640 1 -- 172.21.15.26:0/1055694703 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f871c0052f0 con 0x7f872806f9f0 2024-04-26T21:10:16.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.441+0000 7f8724c00640 1 -- 172.21.15.26:0/1055694703 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c003640 con 0x7f872806f9f0 2024-04-26T21:10:16.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.441+0000 7f872e200640 1 -- 172.21.15.26:0/1055694703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f872813d330 con 0x7f872806f9f0 2024-04-26T21:10:16.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.441+0000 7f872e200640 1 -- 172.21.15.26:0/1055694703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f872813cbb0 con 0x7f872806f9f0 2024-04-26T21:10:16.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.442+0000 7f872e200640 1 -- 172.21.15.26:0/1055694703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87280506c0 con 0x7f872806f9f0 2024-04-26T21:10:16.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.443+0000 7f8724c00640 1 -- 172.21.15.26:0/1055694703 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f871c01a070 con 0x7f872806f9f0 2024-04-26T21:10:16.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.444+0000 7f8724c00640 1 --2- 172.21.15.26:0/1055694703 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f870806c8f0 0x7f870806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:16.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.444+0000 7f8724c00640 1 -- 172.21.15.26:0/1055694703 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f871c08b110 con 0x7f872806f9f0 2024-04-26T21:10:16.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.444+0000 7f8727e00640 1 --2- 172.21.15.26:0/1055694703 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f870806c8f0 0x7f870806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:16.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.445+0000 7f8727e00640 1 --2- 172.21.15.26:0/1055694703 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f870806c8f0 0x7f870806edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8714034000 tx=0x7f871402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:16.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.449+0000 7f8724c00640 1 -- 172.21.15.26:0/1055694703 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f871c0593d0 con 0x7f872806f9f0 2024-04-26T21:10:16.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.652+0000 7f872e200640 1 -- 172.21.15.26:0/1055694703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f87280b0510 con 0x7f872806f9f0 2024-04-26T21:10:16.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.654+0000 7f8724c00640 1 -- 172.21.15.26:0/1055694703 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) ==== 100+0+11 (secure 0 0 0) 0x7f871c058cd0 con 0x7f872806f9f0 2024-04-26T21:10:16.655 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-04-26T21:10:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.657+0000 7f872e200640 1 -- 172.21.15.26:0/1055694703 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f870806c8f0 msgr2=0x7f870806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.657+0000 7f872e200640 1 --2- 172.21.15.26:0/1055694703 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f870806c8f0 0x7f870806edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8714034000 tx=0x7f871402e000 comp rx=0 tx=0).stop 2024-04-26T21:10:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.657+0000 7f872e200640 1 -- 172.21.15.26:0/1055694703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f872806f9f0 msgr2=0x7f872813c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.657+0000 7f872e200640 1 --2- 172.21.15.26:0/1055694703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f872806f9f0 0x7f872813c670 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f871c00c390 tx=0x7f871c00c850 comp rx=0 tx=0).stop 2024-04-26T21:10:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.657+0000 7f872e200640 1 -- 172.21.15.26:0/1055694703 shutdown_connections 2024-04-26T21:10:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.657+0000 7f872e200640 1 --2- 172.21.15.26:0/1055694703 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f870806c8f0 0x7f870806edb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.657+0000 7f872e200640 1 --2- 172.21.15.26:0/1055694703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8728145790 0x7f87280b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.657+0000 7f872e200640 1 --2- 172.21.15.26:0/1055694703 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87280734f0 0x7f87280b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.657+0000 7f872e200640 1 --2- 172.21.15.26:0/1055694703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f872806f9f0 0x7f872813c670 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:16.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.657+0000 7f872e200640 1 -- 172.21.15.26:0/1055694703 >> 172.21.15.26:0/1055694703 conn(0x7f87280cd7d0 msgr2=0x7f872806e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:16.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.658+0000 7f872e200640 1 -- 172.21.15.26:0/1055694703 shutdown_connections 2024-04-26T21:10:16.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.658+0000 7f872e200640 1 -- 172.21.15.26:0/1055694703 wait complete. 2024-04-26T21:10:16.678 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-04-26T21:10:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.892+0000 7f02d5200640 1 Processor -- start 2024-04-26T21:10:16.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.892+0000 7f02d5200640 1 -- start start 2024-04-26T21:10:16.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.894+0000 7f02d5200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f02d006c6f0 0x7f02d006ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:16.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.894+0000 7f02d5200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02d006cb40 con 0x7f02d01469e0 2024-04-26T21:10:16.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.894+0000 7f02d5200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02d00d08f0 con 0x7f02d006c240 2024-04-26T21:10:16.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.894+0000 7f02d5200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f02d00d1500 con 0x7f02d006c6f0 2024-04-26T21:10:16.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.894+0000 7f02cf400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f02d01469e0 0x7f02d015dfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33910/0 (socket says 172.21.15.26:33910) 2024-04-26T21:10:16.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.894+0000 7f02cf400640 1 -- 172.21.15.26:0/3140319556 learned_addr learned my addr 172.21.15.26:0/3140319556 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:16.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.894+0000 7f02cfe00640 1 --2- 172.21.15.26:0/3140319556 >> v2:172.21.15.86:3300/0 conn(0x7f02d006c6f0 0x7f02d006ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:16.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.894+0000 7f02cea00640 1 -- 172.21.15.26:0/3140319556 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3643896711 0 0) 0x7f02d006cb40 con 0x7f02d01469e0 2024-04-26T21:10:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.894+0000 7f02cea00640 1 -- 172.21.15.26:0/3140319556 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02b4003520 con 0x7f02d01469e0 2024-04-26T21:10:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.895+0000 7f02cea00640 1 -- 172.21.15.26:0/3140319556 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 191328137 0 0) 0x7f02b4003520 con 0x7f02d01469e0 2024-04-26T21:10:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.895+0000 7f02cea00640 1 -- 172.21.15.26:0/3140319556 >> v2:172.21.15.86:3300/0 conn(0x7f02d006c6f0 msgr2=0x7f02d006ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.895+0000 7f02cea00640 1 --2- 172.21.15.26:0/3140319556 >> v2:172.21.15.86:3300/0 conn(0x7f02d006c6f0 0x7f02d006ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.895+0000 7f02cea00640 1 -- 172.21.15.26:0/3140319556 >> v1:172.21.15.86:6789/0 conn(0x7f02d006c240 legacy=0x7f02d006f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.895+0000 7f02cea00640 1 -- 172.21.15.26:0/3140319556 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d00d0bd0 con 0x7f02d01469e0 2024-04-26T21:10:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.895+0000 7f02cea00640 1 -- 172.21.15.26:0/3140319556 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f02c0001670 con 0x7f02d01469e0 2024-04-26T21:10:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.895+0000 7f02cea00640 1 -- 172.21.15.26:0/3140319556 >> v1:172.21.15.26:6789/0 conn(0x7f02d01469e0 legacy=0x7f02d015dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.895+0000 7f02cea00640 1 --2- 172.21.15.26:0/3140319556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02b40039b0 0x7f02b4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.895+0000 7f02cea00640 1 --2- 172.21.15.26:0/3140319556 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f02b4004800 0x7f02b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.895+0000 7f02cea00640 1 --2- 172.21.15.26:0/3140319556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b40051c0 0x7f02b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.895+0000 7f02cea00640 1 -- 172.21.15.26:0/3140319556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f02d00d1500 con 0x7f02b40039b0 2024-04-26T21:10:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.895+0000 7f02cea00640 1 -- 172.21.15.26:0/3140319556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02b400c040 con 0x7f02b40051c0 2024-04-26T21:10:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.895+0000 7f02cea00640 1 -- 172.21.15.26:0/3140319556 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f02b400c240 con 0x7f02b4004800 2024-04-26T21:10:16.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.895+0000 7f02cf400640 1 --2- 172.21.15.26:0/3140319556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02b40039b0 0x7f02b4003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:16.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.896+0000 7f02cf400640 1 -- 172.21.15.26:0/3140319556 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f02b4004800 msgr2=0x7f02b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:16.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.896+0000 7f02cf400640 1 --2- 172.21.15.26:0/3140319556 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f02b4004800 0x7f02b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:16.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.896+0000 7f02cf400640 1 -- 172.21.15.26:0/3140319556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b40051c0 msgr2=0x7f02b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:16.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.896+0000 7f02cf400640 1 --2- 172.21.15.26:0/3140319556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02b40051c0 0x7f02b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:16.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.896+0000 7f02cf400640 1 -- 172.21.15.26:0/3140319556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b4003520 con 0x7f02b40039b0 2024-04-26T21:10:16.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.896+0000 7f02cfe00640 1 -- 172.21.15.26:0/3140319556 reap_dead start 2024-04-26T21:10:16.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.897+0000 7f02cf400640 1 --2- 172.21.15.26:0/3140319556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02b40039b0 0x7f02b4003db0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f02c000c240 tx=0x7f02b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a42f7d8dc5adfa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:16.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.897+0000 7f02cea00640 1 -- 172.21.15.26:0/3140319556 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c0007d10 con 0x7f02b40039b0 2024-04-26T21:10:16.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.897+0000 7f02cea00640 1 -- 172.21.15.26:0/3140319556 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c0005ad0 con 0x7f02b40039b0 2024-04-26T21:10:16.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.897+0000 7f02cea00640 1 -- 172.21.15.26:0/3140319556 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c0007070 con 0x7f02b40039b0 2024-04-26T21:10:16.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.897+0000 7f02d5200640 1 -- 172.21.15.26:0/3140319556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02b40039b0 msgr2=0x7f02b4003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:16.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.897+0000 7f02d5200640 1 --2- 172.21.15.26:0/3140319556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02b40039b0 0x7f02b4003db0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f02c000c240 tx=0x7f02b4005830 comp rx=0 tx=0).stop 2024-04-26T21:10:16.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.898+0000 7f02d5200640 1 -- 172.21.15.26:0/3140319556 shutdown_connections 2024-04-26T21:10:16.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.898+0000 7f02d5200640 1 --2- 172.21.15.26:0/3140319556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02b40039b0 0x7f02b4003db0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:16.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.898+0000 7f02d5200640 1 -- 172.21.15.26:0/3140319556 >> 172.21.15.26:0/3140319556 conn(0x7f02d00b3b00 msgr2=0x7f02d00b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:16.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.898+0000 7f02d5200640 1 -- 172.21.15.26:0/3140319556 shutdown_connections 2024-04-26T21:10:16.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.898+0000 7f02d5200640 1 -- 172.21.15.26:0/3140319556 wait complete. 2024-04-26T21:10:16.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.899+0000 7f02d5200640 1 Processor -- start 2024-04-26T21:10:16.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.900+0000 7f02d5200640 1 -- start start 2024-04-26T21:10:16.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.900+0000 7f02d5200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f02b40039b0 0x7f02d0138070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:16.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.901+0000 7f02d5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d01355d0 0x7f02d013a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:16.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.901+0000 7f02d5200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02d0138ac0 0x7f02d0138f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:16.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.901+0000 7f02d5200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f02d00716c0 con 0x7f02d0138ac0 2024-04-26T21:10:16.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.901+0000 7f02d5200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02d00d1190 con 0x7f02d01355d0 2024-04-26T21:10:16.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.901+0000 7f02d5200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f02d0071840 con 0x7f02b40039b0 2024-04-26T21:10:16.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.901+0000 7f02cfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02d0138ac0 0x7f02d0138f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:16.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.901+0000 7f02cfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02d0138ac0 0x7f02d0138f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47122/0 (socket says 172.21.15.26:47122) 2024-04-26T21:10:16.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.901+0000 7f02cfe00640 1 -- 172.21.15.26:0/1277935624 learned_addr learned my addr 172.21.15.26:0/1277935624 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:16.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.901+0000 7f02cf400640 1 --2- 172.21.15.26:0/1277935624 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f02b40039b0 0x7f02d0138070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:16.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.901+0000 7f02c6a00640 1 --2- 172.21.15.26:0/1277935624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d01355d0 0x7f02d013a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:16.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.901+0000 7f02cfe00640 1 -- 172.21.15.26:0/1277935624 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f02b40039b0 msgr2=0x7f02d0138070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:16.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.901+0000 7f02cfe00640 1 --2- 172.21.15.26:0/1277935624 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f02b40039b0 0x7f02d0138070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:16.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.901+0000 7f02cfe00640 1 -- 172.21.15.26:0/1277935624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d01355d0 msgr2=0x7f02d013a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:16.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.901+0000 7f02cfe00640 1 --2- 172.21.15.26:0/1277935624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d01355d0 0x7f02d013a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:16.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.901+0000 7f02cfe00640 1 -- 172.21.15.26:0/1277935624 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d013b460 con 0x7f02d0138ac0 2024-04-26T21:10:16.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.901+0000 7f02c6a00640 1 --2- 172.21.15.26:0/1277935624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d01355d0 0x7f02d013a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:16.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.902+0000 7f02cfe00640 1 --2- 172.21.15.26:0/1277935624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02d0138ac0 0x7f02d0138f50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f02d00690d0 tx=0x7f02c800ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:16.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.902+0000 7f02c7e00640 1 -- 172.21.15.26:0/1277935624 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8018020 con 0x7f02d0138ac0 2024-04-26T21:10:16.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.902+0000 7f02c7e00640 1 -- 172.21.15.26:0/1277935624 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c800bb90 con 0x7f02d0138ac0 2024-04-26T21:10:16.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.902+0000 7f02c7e00640 1 -- 172.21.15.26:0/1277935624 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8014400 con 0x7f02d0138ac0 2024-04-26T21:10:16.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.902+0000 7f02d5200640 1 -- 172.21.15.26:0/1277935624 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d01385b0 con 0x7f02d0138ac0 2024-04-26T21:10:16.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.902+0000 7f02d5200640 1 -- 172.21.15.26:0/1277935624 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d0135a90 con 0x7f02d0138ac0 2024-04-26T21:10:16.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.904+0000 7f02d5200640 1 -- 172.21.15.26:0/1277935624 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02d00693d0 con 0x7f02d0138ac0 2024-04-26T21:10:16.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.904+0000 7f02c7e00640 1 -- 172.21.15.26:0/1277935624 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f02c8005d10 con 0x7f02d0138ac0 2024-04-26T21:10:16.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.905+0000 7f02c7e00640 1 --2- 172.21.15.26:0/1277935624 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f02ac06c8f0 0x7f02ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:16.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.905+0000 7f02c7e00640 1 -- 172.21.15.26:0/1277935624 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f02c808c2f0 con 0x7f02d0138ac0 2024-04-26T21:10:16.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.906+0000 7f02cf400640 1 --2- 172.21.15.26:0/1277935624 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f02ac06c8f0 0x7f02ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:16.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.907+0000 7f02cf400640 1 --2- 172.21.15.26:0/1277935624 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f02ac06c8f0 0x7f02ac06edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f02c0033000 tx=0x7f02c0009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:16.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:16.910+0000 7f02c7e00640 1 -- 172.21.15.26:0/1277935624 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f02c8022030 con 0x7f02d0138ac0 2024-04-26T21:10:17.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.114+0000 7f02d5200640 1 -- 172.21.15.26:0/1277935624 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) -- 0x7f02d006e360 con 0x7f02d0138ac0 2024-04-26T21:10:17.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.115+0000 7f02c7e00640 1 -- 172.21.15.26:0/1277935624 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v37) ==== 97+0+11 (secure 0 0 0) 0x7f02d01385b0 con 0x7f02d0138ac0 2024-04-26T21:10:17.116 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-04-26T21:10:17.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.117+0000 7f02d5200640 1 -- 172.21.15.26:0/1277935624 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f02ac06c8f0 msgr2=0x7f02ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:17.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.117+0000 7f02d5200640 1 --2- 172.21.15.26:0/1277935624 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f02ac06c8f0 0x7f02ac06edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f02c0033000 tx=0x7f02c0009030 comp rx=0 tx=0).stop 2024-04-26T21:10:17.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.117+0000 7f02d5200640 1 -- 172.21.15.26:0/1277935624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02d0138ac0 msgr2=0x7f02d0138f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:17.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.117+0000 7f02d5200640 1 --2- 172.21.15.26:0/1277935624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02d0138ac0 0x7f02d0138f50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f02d00690d0 tx=0x7f02c800ef50 comp rx=0 tx=0).stop 2024-04-26T21:10:17.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.118+0000 7f02d5200640 1 -- 172.21.15.26:0/1277935624 shutdown_connections 2024-04-26T21:10:17.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.118+0000 7f02d5200640 1 --2- 172.21.15.26:0/1277935624 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f02ac06c8f0 0x7f02ac06edb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:17.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.118+0000 7f02d5200640 1 --2- 172.21.15.26:0/1277935624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02d0138ac0 0x7f02d0138f50 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:17.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.118+0000 7f02d5200640 1 --2- 172.21.15.26:0/1277935624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d01355d0 0x7f02d013a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:17.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.118+0000 7f02d5200640 1 --2- 172.21.15.26:0/1277935624 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f02b40039b0 0x7f02d0138070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:17.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.118+0000 7f02d5200640 1 -- 172.21.15.26:0/1277935624 >> 172.21.15.26:0/1277935624 conn(0x7f02d00b3b00 msgr2=0x7f02d015ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:17.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.118+0000 7f02d5200640 1 -- 172.21.15.26:0/1277935624 shutdown_connections 2024-04-26T21:10:17.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.118+0000 7f02d5200640 1 -- 172.21.15.26:0/1277935624 wait complete. 2024-04-26T21:10:17.139 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:10:17.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.347+0000 7f007fe00640 1 Processor -- start 2024-04-26T21:10:17.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.348+0000 7f007fe00640 1 -- start start 2024-04-26T21:10:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.349+0000 7f007fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f00800d1c80 0x7f00800d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.349+0000 7f007fe00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00800b61c0 con 0x7f0080162ba0 2024-04-26T21:10:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.349+0000 7f007fe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00800b64a0 con 0x7f00800d2060 2024-04-26T21:10:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.349+0000 7f007fe00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f00800b5f70 con 0x7f00800d1c80 2024-04-26T21:10:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.350+0000 7f007ea00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f00800d1c80 0x7f00800d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.350+0000 7f007f400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0080162ba0 0x7f00800c8b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33922/0 (socket says 172.21.15.26:33922) 2024-04-26T21:10:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.350+0000 7f007f400640 1 -- 172.21.15.26:0/1958913570 learned_addr learned my addr 172.21.15.26:0/1958913570 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.350+0000 7f007ea00640 1 -- 172.21.15.26:0/1958913570 >> v1:172.21.15.86:6789/0 conn(0x7f00800d2060 legacy=0x7f0080162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.350+0000 7f007ea00640 1 -- 172.21.15.26:0/1958913570 >> v1:172.21.15.26:6789/0 conn(0x7f0080162ba0 legacy=0x7f00800c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.350+0000 7f007ea00640 1 -- 172.21.15.26:0/1958913570 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0080074610 con 0x7f00800d1c80 2024-04-26T21:10:17.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.350+0000 7f007d600640 1 -- 172.21.15.26:0/1958913570 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1602584794 0 0) 0x7f00800b61c0 con 0x7f0080162ba0 2024-04-26T21:10:17.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.351+0000 7f007ea00640 1 --2- 172.21.15.26:0/1958913570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00800d1c80 0x7f00800d3cc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f00700097a0 tx=0x7f007002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd402453075aeac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:17.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.352+0000 7f007d600640 1 -- 172.21.15.26:0/1958913570 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0070037070 con 0x7f00800d1c80 2024-04-26T21:10:17.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.352+0000 7f007d600640 1 -- 172.21.15.26:0/1958913570 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f007002ae50 con 0x7f00800d1c80 2024-04-26T21:10:17.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.352+0000 7f007d600640 1 -- 172.21.15.26:0/1958913570 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0070032660 con 0x7f00800d1c80 2024-04-26T21:10:17.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.352+0000 7f007fe00640 1 -- 172.21.15.26:0/1958913570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00800d1c80 msgr2=0x7f00800d3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:17.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.352+0000 7f007fe00640 1 --2- 172.21.15.26:0/1958913570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00800d1c80 0x7f00800d3cc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f00700097a0 tx=0x7f007002a140 comp rx=0 tx=0).stop 2024-04-26T21:10:17.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.352+0000 7f007fe00640 1 -- 172.21.15.26:0/1958913570 shutdown_connections 2024-04-26T21:10:17.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.352+0000 7f007fe00640 1 --2- 172.21.15.26:0/1958913570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00800d1c80 0x7f00800d3cc0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:17.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.352+0000 7f007fe00640 1 -- 172.21.15.26:0/1958913570 >> 172.21.15.26:0/1958913570 conn(0x7f00800d5020 msgr2=0x7f00800d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.352+0000 7f007fe00640 1 -- 172.21.15.26:0/1958913570 shutdown_connections 2024-04-26T21:10:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.352+0000 7f007fe00640 1 -- 172.21.15.26:0/1958913570 wait complete. 2024-04-26T21:10:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.354+0000 7f007fe00640 1 Processor -- start 2024-04-26T21:10:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.354+0000 7f007fe00640 1 -- start start 2024-04-26T21:10:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.355+0000 7f007fe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f00800d1c80 0x7f00800cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.355+0000 7f007fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00800d2060 0x7f0080154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.355+0000 7f007fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0080162ba0 0x7f00800b9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.355+0000 7f007fe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f00800d0cf0 con 0x7f0080162ba0 2024-04-26T21:10:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.355+0000 7f007fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f00800d3830 con 0x7f00800d2060 2024-04-26T21:10:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.355+0000 7f007fe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f00800d0e70 con 0x7f00800d1c80 2024-04-26T21:10:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.356+0000 7f007f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0080162ba0 0x7f00800b9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.356+0000 7f007f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0080162ba0 0x7f00800b9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47142/0 (socket says 172.21.15.26:47142) 2024-04-26T21:10:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.356+0000 7f007f400640 1 -- 172.21.15.26:0/3652573627 learned_addr learned my addr 172.21.15.26:0/3652573627 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.356+0000 7f007e000640 1 --2- 172.21.15.26:0/3652573627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00800d2060 0x7f0080154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:17.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.356+0000 7f007f400640 1 -- 172.21.15.26:0/3652573627 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f00800d1c80 msgr2=0x7f00800cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:17.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.356+0000 7f007f400640 1 --2- 172.21.15.26:0/3652573627 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f00800d1c80 0x7f00800cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:17.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.356+0000 7f007f400640 1 -- 172.21.15.26:0/3652573627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00800d2060 msgr2=0x7f0080154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:17.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.356+0000 7f007f400640 1 --2- 172.21.15.26:0/3652573627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00800d2060 0x7f0080154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:17.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.356+0000 7f007f400640 1 -- 172.21.15.26:0/3652573627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00800cce70 con 0x7f0080162ba0 2024-04-26T21:10:17.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.357+0000 7f007f400640 1 --2- 172.21.15.26:0/3652573627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0080162ba0 0x7f00800b9130 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f007800c730 tx=0x7f007800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:17.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.357+0000 7f0076a00640 1 -- 172.21.15.26:0/3652573627 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078018070 con 0x7f0080162ba0 2024-04-26T21:10:17.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.357+0000 7f0076a00640 1 -- 172.21.15.26:0/3652573627 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f007800ce40 con 0x7f0080162ba0 2024-04-26T21:10:17.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.357+0000 7f0076a00640 1 -- 172.21.15.26:0/3652573627 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007800a860 con 0x7f0080162ba0 2024-04-26T21:10:17.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.357+0000 7f007fe00640 1 -- 172.21.15.26:0/3652573627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00800cd150 con 0x7f0080162ba0 2024-04-26T21:10:17.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.357+0000 7f007fe00640 1 -- 172.21.15.26:0/3652573627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00800cd710 con 0x7f0080162ba0 2024-04-26T21:10:17.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.358+0000 7f007fe00640 1 -- 172.21.15.26:0/3652573627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f008006a700 con 0x7f0080162ba0 2024-04-26T21:10:17.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.359+0000 7f0076a00640 1 -- 172.21.15.26:0/3652573627 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f007800aa00 con 0x7f0080162ba0 2024-04-26T21:10:17.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.361+0000 7f0076a00640 1 --2- 172.21.15.26:0/3652573627 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f006006c8f0 0x7f006006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:17.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.361+0000 7f0076a00640 1 -- 172.21.15.26:0/3652573627 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f007808b2f0 con 0x7f0080162ba0 2024-04-26T21:10:17.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.361+0000 7f007ea00640 1 --2- 172.21.15.26:0/3652573627 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f006006c8f0 0x7f006006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:17.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.362+0000 7f007ea00640 1 --2- 172.21.15.26:0/3652573627 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f006006c8f0 0x7f006006edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0070034000 tx=0x7f007002c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:17.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.365+0000 7f0076a00640 1 -- 172.21.15.26:0/3652573627 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0078059580 con 0x7f0080162ba0 2024-04-26T21:10:17.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.611+0000 7f007fe00640 1 -- 172.21.15.26:0/3652573627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00801619a0 con 0x7f0080162ba0 2024-04-26T21:10:17.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.612+0000 7f0076a00640 1 -- 172.21.15.26:0/3652573627 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5180 (secure 0 0 0) 0x7f0078058e80 con 0x7f0080162ba0 2024-04-26T21:10:17.614 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:17.614 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2349160700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2349160700},{"type":"v1","addr":"172.21.15.26:6837","nonce":2349160700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4598,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2824191762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2824191762},{"type":"v1","addr":"172.21.15.86:6834","nonce":2824191762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/263746310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":263746310},{"type":"v1","addr":"172.21.15.86:6835","nonce":263746310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:09:48.622025+0000","modified":"2024-04-26T21:10:05.296141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5058":{"gid":5058,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6836/2528180854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2528180854},{"type":"v1","addr":"172.21.15.26:6836","nonce":2528180854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5058,"qdb_cluster":[5058]},"id":2}]} 2024-04-26T21:10:17.614 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 18 2024-04-26T21:10:17.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.614+0000 7f007fe00640 1 -- 172.21.15.26:0/3652573627 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f006006c8f0 msgr2=0x7f006006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:17.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.614+0000 7f007fe00640 1 --2- 172.21.15.26:0/3652573627 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f006006c8f0 0x7f006006edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0070034000 tx=0x7f007002c000 comp rx=0 tx=0).stop 2024-04-26T21:10:17.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.614+0000 7f007fe00640 1 -- 172.21.15.26:0/3652573627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0080162ba0 msgr2=0x7f00800b9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:17.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.614+0000 7f007fe00640 1 --2- 172.21.15.26:0/3652573627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0080162ba0 0x7f00800b9130 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f007800c730 tx=0x7f007800cbf0 comp rx=0 tx=0).stop 2024-04-26T21:10:17.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.614+0000 7f007fe00640 1 -- 172.21.15.26:0/3652573627 shutdown_connections 2024-04-26T21:10:17.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.614+0000 7f007fe00640 1 --2- 172.21.15.26:0/3652573627 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f006006c8f0 0x7f006006edb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:17.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.615+0000 7f007fe00640 1 --2- 172.21.15.26:0/3652573627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0080162ba0 0x7f00800b9130 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:17.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.615+0000 7f007fe00640 1 --2- 172.21.15.26:0/3652573627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00800d2060 0x7f0080154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:17.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.615+0000 7f007fe00640 1 --2- 172.21.15.26:0/3652573627 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f00800d1c80 0x7f00800cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:17.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.615+0000 7f007fe00640 1 -- 172.21.15.26:0/3652573627 >> 172.21.15.26:0/3652573627 conn(0x7f00800d5020 msgr2=0x7f00800d5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:17.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.615+0000 7f007fe00640 1 -- 172.21.15.26:0/3652573627 shutdown_connections 2024-04-26T21:10:17.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.615+0000 7f007fe00640 1 -- 172.21.15.26:0/3652573627 wait complete. 2024-04-26T21:10:17.637 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:10:17.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.849+0000 7f8ab2200640 1 Processor -- start 2024-04-26T21:10:17.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.850+0000 7f8ab2200640 1 -- start start 2024-04-26T21:10:17.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.851+0000 7f8ab2200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8aac0b51b0 0x7f8aac167f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:17.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.851+0000 7f8ab2200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aac0cd5d0 con 0x7f8aac157540 2024-04-26T21:10:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.851+0000 7f8ab2200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aac0cd8b0 con 0x7f8aac1617b0 2024-04-26T21:10:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.852+0000 7f8ab2200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f8aac0cdb90 con 0x7f8aac0b51b0 2024-04-26T21:10:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.852+0000 7f8aab400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8aac157540 0x7f8aac0cf0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33938/0 (socket says 172.21.15.26:33938) 2024-04-26T21:10:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.852+0000 7f8aab400640 1 -- 172.21.15.26:0/3024502210 learned_addr learned my addr 172.21.15.26:0/3024502210 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.852+0000 7f8aabe00640 1 --2- 172.21.15.26:0/3024502210 >> v2:172.21.15.26:3300/0 conn(0x7f8aac0b51b0 0x7f8aac167f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.852+0000 7f8aaa000640 1 -- 172.21.15.26:0/3024502210 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3614788944 0 0) 0x7f8aac0cd5d0 con 0x7f8aac157540 2024-04-26T21:10:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.852+0000 7f8aaa000640 1 -- 172.21.15.26:0/3024502210 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a90003520 con 0x7f8aac157540 2024-04-26T21:10:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.853+0000 7f8aabe00640 1 -- 172.21.15.26:0/3024502210 >> v1:172.21.15.86:6789/0 conn(0x7f8aac1617b0 legacy=0x7f8aac0b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.853+0000 7f8aabe00640 1 -- 172.21.15.26:0/3024502210 >> v1:172.21.15.26:6789/0 conn(0x7f8aac157540 legacy=0x7f8aac0cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.853+0000 7f8aabe00640 1 -- 172.21.15.26:0/3024502210 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aac160cd0 con 0x7f8aac0b51b0 2024-04-26T21:10:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.854+0000 7f8aabe00640 1 --2- 172.21.15.26:0/3024502210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aac0b51b0 0x7f8aac167f30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40097a0 tx=0x7f8aa402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff441779a1b650f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.854+0000 7f8aaa000640 1 -- 172.21.15.26:0/3024502210 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4037070 con 0x7f8aac0b51b0 2024-04-26T21:10:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.854+0000 7f8aaa000640 1 -- 172.21.15.26:0/3024502210 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aa4032440 con 0x7f8aac0b51b0 2024-04-26T21:10:17.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.854+0000 7f8aaa000640 1 -- 172.21.15.26:0/3024502210 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4032810 con 0x7f8aac0b51b0 2024-04-26T21:10:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.855+0000 7f8ab2200640 1 -- 172.21.15.26:0/3024502210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aac0b51b0 msgr2=0x7f8aac167f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.855+0000 7f8ab2200640 1 --2- 172.21.15.26:0/3024502210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aac0b51b0 0x7f8aac167f30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40097a0 tx=0x7f8aa402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:10:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.855+0000 7f8ab2200640 1 -- 172.21.15.26:0/3024502210 shutdown_connections 2024-04-26T21:10:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.855+0000 7f8ab2200640 1 --2- 172.21.15.26:0/3024502210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aac0b51b0 0x7f8aac167f30 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.855+0000 7f8ab2200640 1 -- 172.21.15.26:0/3024502210 >> 172.21.15.26:0/3024502210 conn(0x7f8aac0729a0 msgr2=0x7f8aac06c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.855+0000 7f8ab2200640 1 -- 172.21.15.26:0/3024502210 shutdown_connections 2024-04-26T21:10:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.855+0000 7f8ab2200640 1 -- 172.21.15.26:0/3024502210 wait complete. 2024-04-26T21:10:17.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.856+0000 7f8ab2200640 1 Processor -- start 2024-04-26T21:10:17.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.857+0000 7f8ab2200640 1 -- start start 2024-04-26T21:10:17.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.857+0000 7f8ab2200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aac0b51b0 0x7f8aac0b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:17.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.858+0000 7f8ab2200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8aac157540 0x7f8aac1f8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:17.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.858+0000 7f8ab2200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aac1617b0 0x7f8aac155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:17.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.858+0000 7f8ab2200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac157920 con 0x7f8aac1617b0 2024-04-26T21:10:17.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.858+0000 7f8ab2200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac161320 con 0x7f8aac0b51b0 2024-04-26T21:10:17.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.858+0000 7f8ab2200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f8aac071270 con 0x7f8aac157540 2024-04-26T21:10:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.858+0000 7f8aabe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aac1617b0 0x7f8aac155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.858+0000 7f8aabe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aac1617b0 0x7f8aac155f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47154/0 (socket says 172.21.15.26:47154) 2024-04-26T21:10:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.858+0000 7f8aabe00640 1 -- 172.21.15.26:0/1143664359 learned_addr learned my addr 172.21.15.26:0/1143664359 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.858+0000 7f8aab400640 1 --2- 172.21.15.26:0/1143664359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aac0b51b0 0x7f8aac0b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.858+0000 7f8aaaa00640 1 --2- 172.21.15.26:0/1143664359 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8aac157540 0x7f8aac1f8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.858+0000 7f8aabe00640 1 -- 172.21.15.26:0/1143664359 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8aac157540 msgr2=0x7f8aac1f8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.858+0000 7f8aabe00640 1 --2- 172.21.15.26:0/1143664359 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8aac157540 0x7f8aac1f8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.858+0000 7f8aabe00640 1 -- 172.21.15.26:0/1143664359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aac0b51b0 msgr2=0x7f8aac0b7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.858+0000 7f8aabe00640 1 --2- 172.21.15.26:0/1143664359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aac0b51b0 0x7f8aac0b7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.858+0000 7f8aabe00640 1 -- 172.21.15.26:0/1143664359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aac0b8350 con 0x7f8aac1617b0 2024-04-26T21:10:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.858+0000 7f8aaaa00640 1 --2- 172.21.15.26:0/1143664359 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8aac157540 0x7f8aac1f8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.858+0000 7f8aab400640 1 --2- 172.21.15.26:0/1143664359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aac0b51b0 0x7f8aac0b7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:17.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.859+0000 7f8aabe00640 1 --2- 172.21.15.26:0/1143664359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aac1617b0 0x7f8aac155f30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4035040 tx=0x7f8aa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:17.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.859+0000 7f8aa3400640 1 -- 172.21.15.26:0/1143664359 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4037040 con 0x7f8aac1617b0 2024-04-26T21:10:17.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.859+0000 7f8aa3400640 1 -- 172.21.15.26:0/1143664359 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aa4042070 con 0x7f8aac1617b0 2024-04-26T21:10:17.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.859+0000 7f8aa3400640 1 -- 172.21.15.26:0/1143664359 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa40308a0 con 0x7f8aac1617b0 2024-04-26T21:10:17.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.859+0000 7f8ab2200640 1 -- 172.21.15.26:0/1143664359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aac0b6d70 con 0x7f8aac1617b0 2024-04-26T21:10:17.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.859+0000 7f8ab2200640 1 -- 172.21.15.26:0/1143664359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aac0b7330 con 0x7f8aac1617b0 2024-04-26T21:10:17.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.860+0000 7f8ab2200640 1 -- 172.21.15.26:0/1143664359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aac06c770 con 0x7f8aac1617b0 2024-04-26T21:10:17.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.861+0000 7f8aa3400640 1 -- 172.21.15.26:0/1143664359 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8aa4030a40 con 0x7f8aac1617b0 2024-04-26T21:10:17.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.862+0000 7f8aa3400640 1 --2- 172.21.15.26:0/1143664359 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8a8806c8f0 0x7f8a8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:17.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.862+0000 7f8aa3400640 1 -- 172.21.15.26:0/1143664359 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f8aa40ac640 con 0x7f8aac1617b0 2024-04-26T21:10:17.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.863+0000 7f8aab400640 1 --2- 172.21.15.26:0/1143664359 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8a8806c8f0 0x7f8a8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:17.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.863+0000 7f8aab400640 1 --2- 172.21.15.26:0/1143664359 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8a8806c8f0 0x7f8a8806edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c001560 tx=0x7f8a9c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:17.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:17.867+0000 7f8aa3400640 1 -- 172.21.15.26:0/1143664359 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8aac06c770 con 0x7f8aac1617b0 2024-04-26T21:10:18.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.073+0000 7f8ab2200640 1 -- 172.21.15.26:0/1143664359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8aac0b33f0 con 0x7f8aac1617b0 2024-04-26T21:10:18.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.076+0000 7f8aa3400640 1 -- 172.21.15.26:0/1143664359 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+18433 (secure 0 0 0) 0x7f8aa407a8d0 con 0x7f8aac1617b0 2024-04-26T21:10:18.079 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:18.079 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":37,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:10:06.822858+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:46.646256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:47.613883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-04-26T21:10:04.249730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:10:18.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.079+0000 7f8ab2200640 1 -- 172.21.15.26:0/1143664359 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8a8806c8f0 msgr2=0x7f8a8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:18.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.079+0000 7f8ab2200640 1 --2- 172.21.15.26:0/1143664359 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8a8806c8f0 0x7f8a8806edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c001560 tx=0x7f8a9c009040 comp rx=0 tx=0).stop 2024-04-26T21:10:18.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.079+0000 7f8ab2200640 1 -- 172.21.15.26:0/1143664359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aac1617b0 msgr2=0x7f8aac155f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:18.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.079+0000 7f8ab2200640 1 --2- 172.21.15.26:0/1143664359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aac1617b0 0x7f8aac155f30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4035040 tx=0x7f8aa40046c0 comp rx=0 tx=0).stop 2024-04-26T21:10:18.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.079+0000 7f8ab2200640 1 -- 172.21.15.26:0/1143664359 shutdown_connections 2024-04-26T21:10:18.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.079+0000 7f8ab2200640 1 --2- 172.21.15.26:0/1143664359 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f8a8806c8f0 0x7f8a8806edb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:18.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.079+0000 7f8ab2200640 1 --2- 172.21.15.26:0/1143664359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aac1617b0 0x7f8aac155f30 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:18.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.079+0000 7f8ab2200640 1 --2- 172.21.15.26:0/1143664359 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8aac157540 0x7f8aac1f8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:18.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.079+0000 7f8ab2200640 1 --2- 172.21.15.26:0/1143664359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aac0b51b0 0x7f8aac0b7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:18.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.079+0000 7f8ab2200640 1 -- 172.21.15.26:0/1143664359 >> 172.21.15.26:0/1143664359 conn(0x7f8aac0729a0 msgr2=0x7f8aac0d0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:18.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.079+0000 7f8ab2200640 1 -- 172.21.15.26:0/1143664359 shutdown_connections 2024-04-26T21:10:18.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.079+0000 7f8ab2200640 1 -- 172.21.15.26:0/1143664359 wait complete. 2024-04-26T21:10:18.098 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:10:18.098 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:10:18.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.307+0000 7faf17800640 1 Processor -- start 2024-04-26T21:10:18.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.307+0000 7faf17800640 1 -- start start 2024-04-26T21:10:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.309+0000 7faf17800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf100b6a40 0x7faf100b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.309+0000 7faf17800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf1015bbc0 con 0x7faf100cf1c0 2024-04-26T21:10:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.309+0000 7faf17800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf100d09b0 con 0x7faf10072ac0 2024-04-26T21:10:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.309+0000 7faf17800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7faf1006eec0 con 0x7faf100b6a40 2024-04-26T21:10:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.309+0000 7faf14e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faf100cf1c0 0x7faf1006f350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33940/0 (socket says 172.21.15.26:33940) 2024-04-26T21:10:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.309+0000 7faf14e00640 1 -- 172.21.15.26:0/2732613260 learned_addr learned my addr 172.21.15.26:0/2732613260 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.309+0000 7faf0fe00640 1 --2- 172.21.15.26:0/2732613260 >> v2:172.21.15.26:3300/0 conn(0x7faf100b6a40 0x7faf100b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.310+0000 7faf0f400640 1 -- 172.21.15.26:0/2732613260 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1461110481 0 0) 0x7faf1015bbc0 con 0x7faf100cf1c0 2024-04-26T21:10:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.310+0000 7faf0f400640 1 -- 172.21.15.26:0/2732613260 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faef4003520 con 0x7faf100cf1c0 2024-04-26T21:10:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.310+0000 7faf0fe00640 1 -- 172.21.15.26:0/2732613260 >> v1:172.21.15.86:6789/0 conn(0x7faf10072ac0 legacy=0x7faf100cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.310+0000 7faf0fe00640 1 -- 172.21.15.26:0/2732613260 >> v1:172.21.15.26:6789/0 conn(0x7faf100cf1c0 legacy=0x7faf1006f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.310+0000 7faf0fe00640 1 -- 172.21.15.26:0/2732613260 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf100d0240 con 0x7faf100b6a40 2024-04-26T21:10:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.311+0000 7faf0fe00640 1 --2- 172.21.15.26:0/2732613260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf100b6a40 0x7faf100b4030 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7faef8009d90 tx=0x7faef802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d80592534b85541 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.311+0000 7faf0f400640 1 -- 172.21.15.26:0/2732613260 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8031600 con 0x7faf100b6a40 2024-04-26T21:10:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.311+0000 7faf0f400640 1 -- 172.21.15.26:0/2732613260 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faef80317a0 con 0x7faf100b6a40 2024-04-26T21:10:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.311+0000 7faf0f400640 1 -- 172.21.15.26:0/2732613260 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8031b70 con 0x7faf100b6a40 2024-04-26T21:10:18.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.312+0000 7faf17800640 1 -- 172.21.15.26:0/2732613260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf100b6a40 msgr2=0x7faf100b4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:18.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.312+0000 7faf17800640 1 --2- 172.21.15.26:0/2732613260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf100b6a40 0x7faf100b4030 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7faef8009d90 tx=0x7faef802a610 comp rx=0 tx=0).stop 2024-04-26T21:10:18.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.312+0000 7faf17800640 1 -- 172.21.15.26:0/2732613260 shutdown_connections 2024-04-26T21:10:18.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.312+0000 7faf17800640 1 --2- 172.21.15.26:0/2732613260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf100b6a40 0x7faf100b4030 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:18.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.312+0000 7faf17800640 1 -- 172.21.15.26:0/2732613260 >> 172.21.15.26:0/2732613260 conn(0x7faf100cd840 msgr2=0x7faf100cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:18.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.312+0000 7faf17800640 1 -- 172.21.15.26:0/2732613260 shutdown_connections 2024-04-26T21:10:18.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.312+0000 7faf17800640 1 -- 172.21.15.26:0/2732613260 wait complete. 2024-04-26T21:10:18.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.315+0000 7faf17800640 1 Processor -- start 2024-04-26T21:10:18.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.315+0000 7faf17800640 1 -- start start 2024-04-26T21:10:18.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.316+0000 7faf17800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf10072ac0 0x7faf10146900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.316+0000 7faf17800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf100b6a40 0x7faf1014ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.316+0000 7faf17800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf100cf1c0 0x7faf1014c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.316+0000 7faf17800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faf1006fbe0 con 0x7faf10072ac0 2024-04-26T21:10:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.316+0000 7faf17800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf1006fa60 con 0x7faf100cf1c0 2024-04-26T21:10:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.316+0000 7faf17800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7faf100cfcb0 con 0x7faf100b6a40 2024-04-26T21:10:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.316+0000 7faf14e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf10072ac0 0x7faf10146900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.316+0000 7faf14e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf10072ac0 0x7faf10146900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47168/0 (socket says 172.21.15.26:47168) 2024-04-26T21:10:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.316+0000 7faf14e00640 1 -- 172.21.15.26:0/3199544775 learned_addr learned my addr 172.21.15.26:0/3199544775 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.317+0000 7faf14e00640 1 -- 172.21.15.26:0/3199544775 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf100b6a40 msgr2=0x7faf1014ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.317+0000 7faf14e00640 1 --2- 172.21.15.26:0/3199544775 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf100b6a40 0x7faf1014ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.317+0000 7faf14e00640 1 -- 172.21.15.26:0/3199544775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf100cf1c0 msgr2=0x7faf1014c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.317+0000 7faf14e00640 1 --2- 172.21.15.26:0/3199544775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf100cf1c0 0x7faf1014c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.317+0000 7faf14e00640 1 -- 172.21.15.26:0/3199544775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1014f120 con 0x7faf10072ac0 2024-04-26T21:10:18.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.317+0000 7faf14e00640 1 --2- 172.21.15.26:0/3199544775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf10072ac0 0x7faf10146900 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7faf0000c3c0 tx=0x7faf0000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:18.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.317+0000 7faf0cc00640 1 -- 172.21.15.26:0/3199544775 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf00017070 con 0x7faf10072ac0 2024-04-26T21:10:18.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.317+0000 7faf0cc00640 1 -- 172.21.15.26:0/3199544775 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf00005ce0 con 0x7faf10072ac0 2024-04-26T21:10:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.317+0000 7faf0cc00640 1 -- 172.21.15.26:0/3199544775 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf000072f0 con 0x7faf10072ac0 2024-04-26T21:10:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.317+0000 7faf17800640 1 -- 172.21.15.26:0/3199544775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1014f400 con 0x7faf10072ac0 2024-04-26T21:10:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.318+0000 7faf17800640 1 -- 172.21.15.26:0/3199544775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf1014f9c0 con 0x7faf10072ac0 2024-04-26T21:10:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.319+0000 7faf17800640 1 -- 172.21.15.26:0/3199544775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf1006a700 con 0x7faf10072ac0 2024-04-26T21:10:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.320+0000 7faf0cc00640 1 -- 172.21.15.26:0/3199544775 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf0001a070 con 0x7faf10072ac0 2024-04-26T21:10:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.321+0000 7faf0cc00640 1 --2- 172.21.15.26:0/3199544775 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7faeec06c8f0 0x7faeec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.321+0000 7faf0cc00640 1 -- 172.21.15.26:0/3199544775 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7faf0008b130 con 0x7faf10072ac0 2024-04-26T21:10:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.321+0000 7faf0fe00640 1 --2- 172.21.15.26:0/3199544775 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7faeec06c8f0 0x7faeec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:18.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.322+0000 7faf0fe00640 1 --2- 172.21.15.26:0/3199544775 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7faeec06c8f0 0x7faeec06edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faef8034000 tx=0x7faef802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:18.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.326+0000 7faf0cc00640 1 -- 172.21.15.26:0/3199544775 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faf000593f0 con 0x7faf10072ac0 2024-04-26T21:10:18.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.569+0000 7faf17800640 1 -- 172.21.15.26:0/3199544775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf10146f00 con 0x7faf10072ac0 2024-04-26T21:10:18.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.570+0000 7faf0cc00640 1 -- 172.21.15.26:0/3199544775 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5180 (secure 0 0 0) 0x7faf00058cf0 con 0x7faf10072ac0 2024-04-26T21:10:18.572 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:18.572 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2349160700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2349160700},{"type":"v1","addr":"172.21.15.26:6837","nonce":2349160700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4598,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2824191762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2824191762},{"type":"v1","addr":"172.21.15.86:6834","nonce":2824191762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/263746310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":263746310},{"type":"v1","addr":"172.21.15.86:6835","nonce":263746310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:09:48.622025+0000","modified":"2024-04-26T21:10:05.296141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5058":{"gid":5058,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6836/2528180854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2528180854},{"type":"v1","addr":"172.21.15.26:6836","nonce":2528180854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5058,"qdb_cluster":[5058]},"id":2}]} 2024-04-26T21:10:18.572 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 18 2024-04-26T21:10:18.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.572+0000 7faf17800640 1 -- 172.21.15.26:0/3199544775 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7faeec06c8f0 msgr2=0x7faeec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:18.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.572+0000 7faf17800640 1 --2- 172.21.15.26:0/3199544775 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7faeec06c8f0 0x7faeec06edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faef8034000 tx=0x7faef802e000 comp rx=0 tx=0).stop 2024-04-26T21:10:18.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.572+0000 7faf17800640 1 -- 172.21.15.26:0/3199544775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf10072ac0 msgr2=0x7faf10146900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:18.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.572+0000 7faf17800640 1 --2- 172.21.15.26:0/3199544775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf10072ac0 0x7faf10146900 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7faf0000c3c0 tx=0x7faf0000c880 comp rx=0 tx=0).stop 2024-04-26T21:10:18.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.573+0000 7faf17800640 1 -- 172.21.15.26:0/3199544775 shutdown_connections 2024-04-26T21:10:18.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.573+0000 7faf17800640 1 --2- 172.21.15.26:0/3199544775 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7faeec06c8f0 0x7faeec06edb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:18.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.573+0000 7faf17800640 1 --2- 172.21.15.26:0/3199544775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf100cf1c0 0x7faf1014c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:18.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.573+0000 7faf17800640 1 --2- 172.21.15.26:0/3199544775 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf100b6a40 0x7faf1014ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:18.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.573+0000 7faf17800640 1 --2- 172.21.15.26:0/3199544775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf10072ac0 0x7faf10146900 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:18.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.573+0000 7faf17800640 1 -- 172.21.15.26:0/3199544775 >> 172.21.15.26:0/3199544775 conn(0x7faf100cd840 msgr2=0x7faf10153c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:18.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.573+0000 7faf17800640 1 -- 172.21.15.26:0/3199544775 shutdown_connections 2024-04-26T21:10:18.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.573+0000 7faf17800640 1 -- 172.21.15.26:0/3199544775 wait complete. 2024-04-26T21:10:18.594 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:10:18.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.798+0000 7fc7ba400640 1 Processor -- start 2024-04-26T21:10:18.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.799+0000 7fc7ba400640 1 -- start start 2024-04-26T21:10:18.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.800+0000 7fc7ba400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc7b4162ba0 0x7fc7b40d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.800+0000 7fc7ba400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc7b40d2060 0x7fc7b40d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.800+0000 7fc7ba400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7b40b61c0 con 0x7fc7b40d1c80 2024-04-26T21:10:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.800+0000 7fc7ba400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fc7b40b5f70 con 0x7fc7b40d2060 2024-04-26T21:10:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.800+0000 7fc7ba400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc7b40d3830 con 0x7fc7b4162ba0 2024-04-26T21:10:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.801+0000 7fc7b9a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc7b40d2060 0x7fc7b40d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.801+0000 7fc7b9a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc7b40d2060 0x7fc7b40d3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47180/0 (socket says 172.21.15.26:47180) 2024-04-26T21:10:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.801+0000 7fc7b9a00640 1 -- 172.21.15.26:0/1045906649 learned_addr learned my addr 172.21.15.26:0/1045906649 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.801+0000 7fc7abe00640 1 --2- 172.21.15.26:0/1045906649 >> v2:172.21.15.86:3300/0 conn(0x7fc7b4162ba0 0x7fc7b40d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.801+0000 7fc7b9a00640 1 -- 172.21.15.26:0/1045906649 >> v2:172.21.15.86:3300/0 conn(0x7fc7b4162ba0 msgr2=0x7fc7b40d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.801+0000 7fc7b9a00640 1 --2- 172.21.15.26:0/1045906649 >> v2:172.21.15.86:3300/0 conn(0x7fc7b4162ba0 0x7fc7b40d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.801+0000 7fc7b9a00640 1 -- 172.21.15.26:0/1045906649 >> v1:172.21.15.86:6789/0 conn(0x7fc7b40d1c80 legacy=0x7fc7b4162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.801+0000 7fc7b9a00640 1 -- 172.21.15.26:0/1045906649 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b40b64a0 con 0x7fc7b40d2060 2024-04-26T21:10:18.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.802+0000 7fc7b9a00640 1 --2- 172.21.15.26:0/1045906649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b40d2060 0x7fc7b40d3cc0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40d4ff0 tx=0x7fc7ac02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7300185c4af071 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:18.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.802+0000 7fc7b3e00640 1 -- 172.21.15.26:0/1045906649 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac036070 con 0x7fc7b40d2060 2024-04-26T21:10:18.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.803+0000 7fc7b3e00640 1 -- 172.21.15.26:0/1045906649 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7ac031600 con 0x7fc7b40d2060 2024-04-26T21:10:18.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.803+0000 7fc7b3e00640 1 -- 172.21.15.26:0/1045906649 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac0319d0 con 0x7fc7b40d2060 2024-04-26T21:10:18.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.803+0000 7fc7ba400640 1 -- 172.21.15.26:0/1045906649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b40d2060 msgr2=0x7fc7b40d3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:18.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.803+0000 7fc7ba400640 1 --2- 172.21.15.26:0/1045906649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b40d2060 0x7fc7b40d3cc0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40d4ff0 tx=0x7fc7ac02a660 comp rx=0 tx=0).stop 2024-04-26T21:10:18.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.803+0000 7fc7ba400640 1 -- 172.21.15.26:0/1045906649 shutdown_connections 2024-04-26T21:10:18.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.803+0000 7fc7ba400640 1 --2- 172.21.15.26:0/1045906649 >> v2:172.21.15.86:3300/0 conn(0x7fc7b4162ba0 0x7fc7b40d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:18.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.803+0000 7fc7ba400640 1 --2- 172.21.15.26:0/1045906649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b40d2060 0x7fc7b40d3cc0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:18.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.803+0000 7fc7ba400640 1 -- 172.21.15.26:0/1045906649 >> 172.21.15.26:0/1045906649 conn(0x7fc7b40d5020 msgr2=0x7fc7b40d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:18.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.804+0000 7fc7ba400640 1 -- 172.21.15.26:0/1045906649 shutdown_connections 2024-04-26T21:10:18.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.804+0000 7fc7ba400640 1 -- 172.21.15.26:0/1045906649 wait complete. 2024-04-26T21:10:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.806+0000 7fc7ba400640 1 Processor -- start 2024-04-26T21:10:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.806+0000 7fc7ba400640 1 -- start start 2024-04-26T21:10:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.807+0000 7fc7ba400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc7b40d1c80 0x7fc7b40ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.807+0000 7fc7ba400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7b40d2060 0x7fc7b4150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.807+0000 7fc7ba400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b4162ba0 0x7fc7b40b9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.807+0000 7fc7ba400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b40d5cc0 con 0x7fc7b4162ba0 2024-04-26T21:10:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.807+0000 7fc7ba400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b40d5b40 con 0x7fc7b40d2060 2024-04-26T21:10:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.807+0000 7fc7ba400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b40c8b70 con 0x7fc7b40d1c80 2024-04-26T21:10:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.808+0000 7fc7b9a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b4162ba0 0x7fc7b40b9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.808+0000 7fc7b9a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b4162ba0 0x7fc7b40b9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47184/0 (socket says 172.21.15.26:47184) 2024-04-26T21:10:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.808+0000 7fc7b9a00640 1 -- 172.21.15.26:0/4182265864 learned_addr learned my addr 172.21.15.26:0/4182265864 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.808+0000 7fc7abe00640 1 --2- 172.21.15.26:0/4182265864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7b40d2060 0x7fc7b4150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.808+0000 7fc7b9a00640 1 -- 172.21.15.26:0/4182265864 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc7b40d1c80 msgr2=0x7fc7b40ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.808+0000 7fc7b9a00640 1 --2- 172.21.15.26:0/4182265864 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc7b40d1c80 0x7fc7b40ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.808+0000 7fc7b9a00640 1 -- 172.21.15.26:0/4182265864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7b40d2060 msgr2=0x7fc7b4150470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.808+0000 7fc7b9a00640 1 --2- 172.21.15.26:0/4182265864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7b40d2060 0x7fc7b4150470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.808+0000 7fc7b9a00640 1 -- 172.21.15.26:0/4182265864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b41509b0 con 0x7fc7b4162ba0 2024-04-26T21:10:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.808+0000 7fc7b9a00640 1 --2- 172.21.15.26:0/4182265864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b4162ba0 0x7fc7b40b9dd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac02ac50 tx=0x7fc7ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.809+0000 7fc7b1600640 1 -- 172.21.15.26:0/4182265864 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac03f020 con 0x7fc7b4162ba0 2024-04-26T21:10:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.809+0000 7fc7b1600640 1 -- 172.21.15.26:0/4182265864 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7ac037040 con 0x7fc7b4162ba0 2024-04-26T21:10:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.809+0000 7fc7ba400640 1 -- 172.21.15.26:0/4182265864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b40cc190 con 0x7fc7b4162ba0 2024-04-26T21:10:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.809+0000 7fc7b1600640 1 -- 172.21.15.26:0/4182265864 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac036070 con 0x7fc7b4162ba0 2024-04-26T21:10:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.809+0000 7fc7ba400640 1 -- 172.21.15.26:0/4182265864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b40cc710 con 0x7fc7b4162ba0 2024-04-26T21:10:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.810+0000 7fc7ba400640 1 -- 172.21.15.26:0/4182265864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7b406a700 con 0x7fc7b4162ba0 2024-04-26T21:10:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.811+0000 7fc7b1600640 1 -- 172.21.15.26:0/4182265864 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7ac042020 con 0x7fc7b4162ba0 2024-04-26T21:10:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.813+0000 7fc7b1600640 1 --2- 172.21.15.26:0/4182265864 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc79006c8f0 0x7fc79006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.813+0000 7fc7b1600640 1 -- 172.21.15.26:0/4182265864 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7fc7ac0ac1c0 con 0x7fc7b4162ba0 2024-04-26T21:10:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.813+0000 7fc7b9000640 1 --2- 172.21.15.26:0/4182265864 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc79006c8f0 0x7fc79006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.814+0000 7fc7b9000640 1 --2- 172.21.15.26:0/4182265864 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc79006c8f0 0x7fc79006edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4003cc0 tx=0x7fc7a40079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:18.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:18.817+0000 7fc7b1600640 1 -- 172.21.15.26:0/4182265864 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc7ac049030 con 0x7fc7b4162ba0 2024-04-26T21:10:19.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.051+0000 7fc7ba400640 1 -- 172.21.15.26:0/4182265864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7b40d32f0 con 0x7fc7b4162ba0 2024-04-26T21:10:19.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.051+0000 7fc7b1600640 1 -- 172.21.15.26:0/4182265864 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5180 (secure 0 0 0) 0x7fc7b41509b0 con 0x7fc7b4162ba0 2024-04-26T21:10:19.053 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:19.053 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2349160700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2349160700},{"type":"v1","addr":"172.21.15.26:6837","nonce":2349160700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4598,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2824191762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2824191762},{"type":"v1","addr":"172.21.15.86:6834","nonce":2824191762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/263746310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":263746310},{"type":"v1","addr":"172.21.15.86:6835","nonce":263746310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T21:09:48.622025+0000","modified":"2024-04-26T21:10:05.296141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5058},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5058":{"gid":5058,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.26:6836/2528180854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":2528180854},{"type":"v1","addr":"172.21.15.26:6836","nonce":2528180854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5058,"qdb_cluster":[5058]},"id":2}]} 2024-04-26T21:10:19.053 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 18 2024-04-26T21:10:19.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.054+0000 7fc7ba400640 1 -- 172.21.15.26:0/4182265864 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc79006c8f0 msgr2=0x7fc79006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:19.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.054+0000 7fc7ba400640 1 --2- 172.21.15.26:0/4182265864 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc79006c8f0 0x7fc79006edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4003cc0 tx=0x7fc7a40079b0 comp rx=0 tx=0).stop 2024-04-26T21:10:19.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.054+0000 7fc7ba400640 1 -- 172.21.15.26:0/4182265864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b4162ba0 msgr2=0x7fc7b40b9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:19.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.054+0000 7fc7ba400640 1 --2- 172.21.15.26:0/4182265864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b4162ba0 0x7fc7b40b9dd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac02ac50 tx=0x7fc7ac004a40 comp rx=0 tx=0).stop 2024-04-26T21:10:19.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.054+0000 7fc7ba400640 1 -- 172.21.15.26:0/4182265864 shutdown_connections 2024-04-26T21:10:19.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.054+0000 7fc7ba400640 1 --2- 172.21.15.26:0/4182265864 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fc79006c8f0 0x7fc79006edb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:19.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.054+0000 7fc7ba400640 1 --2- 172.21.15.26:0/4182265864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7b4162ba0 0x7fc7b40b9dd0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:19.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.054+0000 7fc7ba400640 1 --2- 172.21.15.26:0/4182265864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7b40d2060 0x7fc7b4150470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:19.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.054+0000 7fc7ba400640 1 --2- 172.21.15.26:0/4182265864 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc7b40d1c80 0x7fc7b40ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:19.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.054+0000 7fc7ba400640 1 -- 172.21.15.26:0/4182265864 >> 172.21.15.26:0/4182265864 conn(0x7fc7b40d5020 msgr2=0x7fc7b415fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:19.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.054+0000 7fc7ba400640 1 -- 172.21.15.26:0/4182265864 shutdown_connections 2024-04-26T21:10:19.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.054+0000 7fc7ba400640 1 -- 172.21.15.26:0/4182265864 wait complete. 2024-04-26T21:10:19.074 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:10:19.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.275+0000 7fb9f1e00640 1 Processor -- start 2024-04-26T21:10:19.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.276+0000 7fb9f1e00640 1 -- start start 2024-04-26T21:10:19.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.277+0000 7fb9f1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb9ec15dcb0 0x7fb9ec164590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:19.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.278+0000 7fb9f1e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9ec06ebb0 0x7fb9ec0d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:19.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.278+0000 7fb9f1e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9ec164ad0 con 0x7fb9ec06e700 2024-04-26T21:10:19.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.278+0000 7fb9f1e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb9ec15e090 con 0x7fb9ec15dcb0 2024-04-26T21:10:19.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.278+0000 7fb9f1e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb9ec072500 con 0x7fb9ec06ebb0 2024-04-26T21:10:19.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.278+0000 7fb9ebe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb9ec06e700 0x7fb9ec1677a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54558/0 (socket says 172.21.15.26:54558) 2024-04-26T21:10:19.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.278+0000 7fb9ebe00640 1 -- 172.21.15.26:0/3231641203 learned_addr learned my addr 172.21.15.26:0/3231641203 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:19.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.278+0000 7fb9f0a00640 1 --2- 172.21.15.26:0/3231641203 >> v2:172.21.15.86:3300/0 conn(0x7fb9ec15dcb0 0x7fb9ec164590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:19.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.278+0000 7fb9f1400640 1 --2- 172.21.15.26:0/3231641203 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9ec06ebb0 0x7fb9ec0d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:19.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.279+0000 7fb9eb400640 1 -- 172.21.15.26:0/3231641203 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1583329296 0 0) 0x7fb9ec164ad0 con 0x7fb9ec06e700 2024-04-26T21:10:19.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.279+0000 7fb9eb400640 1 -- 172.21.15.26:0/3231641203 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9d0003520 con 0x7fb9ec06e700 2024-04-26T21:10:19.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.279+0000 7fb9eb400640 1 -- 172.21.15.26:0/3231641203 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 369530661 0 0) 0x7fb9d0003520 con 0x7fb9ec06e700 2024-04-26T21:10:19.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.279+0000 7fb9eb400640 1 -- 172.21.15.26:0/3231641203 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9ec06ebb0 msgr2=0x7fb9ec0d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:19.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.279+0000 7fb9eb400640 1 --2- 172.21.15.26:0/3231641203 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9ec06ebb0 0x7fb9ec0d1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:19.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.279+0000 7fb9eb400640 1 -- 172.21.15.26:0/3231641203 >> v2:172.21.15.86:3300/0 conn(0x7fb9ec15dcb0 msgr2=0x7fb9ec164590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:19.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.279+0000 7fb9eb400640 1 --2- 172.21.15.26:0/3231641203 >> v2:172.21.15.86:3300/0 conn(0x7fb9ec15dcb0 0x7fb9ec164590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:19.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.279+0000 7fb9eb400640 1 -- 172.21.15.26:0/3231641203 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9ec0727e0 con 0x7fb9ec06e700 2024-04-26T21:10:19.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.279+0000 7fb9eb400640 1 -- 172.21.15.26:0/3231641203 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7fb9d8001670 con 0x7fb9ec06e700 2024-04-26T21:10:19.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.279+0000 7fb9eb400640 1 -- 172.21.15.26:0/3231641203 >> v1:172.21.15.26:6789/0 conn(0x7fb9ec06e700 legacy=0x7fb9ec1677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:19.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.279+0000 7fb9eb400640 1 --2- 172.21.15.26:0/3231641203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9d00039b0 0x7fb9d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:19.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.279+0000 7fb9eb400640 1 --2- 172.21.15.26:0/3231641203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9d0004800 0x7fb9d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:19.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.279+0000 7fb9eb400640 1 --2- 172.21.15.26:0/3231641203 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9ec06ebb0 0x7fb9d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:19.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.279+0000 7fb9eb400640 1 -- 172.21.15.26:0/3231641203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ec072500 con 0x7fb9d00039b0 2024-04-26T21:10:19.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.279+0000 7fb9eb400640 1 -- 172.21.15.26:0/3231641203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d000ba70 con 0x7fb9d0004800 2024-04-26T21:10:19.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.279+0000 7fb9eb400640 1 -- 172.21.15.26:0/3231641203 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb9d000bc20 con 0x7fb9ec06ebb0 2024-04-26T21:10:19.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.279+0000 7fb9f0a00640 1 --2- 172.21.15.26:0/3231641203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9d00039b0 0x7fb9d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:19.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.280+0000 7fb9f1400640 1 --2- 172.21.15.26:0/3231641203 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9ec06ebb0 0x7fb9d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:19.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.280+0000 7fb9ebe00640 1 --2- 172.21.15.26:0/3231641203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9d0004800 0x7fb9d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:19.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.280+0000 7fb9f0a00640 1 -- 172.21.15.26:0/3231641203 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9ec06ebb0 msgr2=0x7fb9d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:19.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.280+0000 7fb9f0a00640 1 --2- 172.21.15.26:0/3231641203 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9ec06ebb0 0x7fb9d00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:19.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.280+0000 7fb9f0a00640 1 -- 172.21.15.26:0/3231641203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9d0004800 msgr2=0x7fb9d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:19.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.280+0000 7fb9f0a00640 1 --2- 172.21.15.26:0/3231641203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9d0004800 0x7fb9d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:19.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.280+0000 7fb9f0a00640 1 -- 172.21.15.26:0/3231641203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d0003520 con 0x7fb9d00039b0 2024-04-26T21:10:19.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.280+0000 7fb9ebe00640 1 --2- 172.21.15.26:0/3231641203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9d0004800 0x7fb9d0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:19.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.281+0000 7fb9f0a00640 1 --2- 172.21.15.26:0/3231641203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9d00039b0 0x7fb9d0003dd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc009a90 tx=0x7fb9d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e3327cb683fe3e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:19.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.281+0000 7fb9eb400640 1 -- 172.21.15.26:0/3231641203 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc036070 con 0x7fb9d00039b0 2024-04-26T21:10:19.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.281+0000 7fb9eb400640 1 -- 172.21.15.26:0/3231641203 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9dc031400 con 0x7fb9d00039b0 2024-04-26T21:10:19.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.281+0000 7fb9eb400640 1 -- 172.21.15.26:0/3231641203 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc009690 con 0x7fb9d00039b0 2024-04-26T21:10:19.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.282+0000 7fb9f1e00640 1 -- 172.21.15.26:0/3231641203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9d00039b0 msgr2=0x7fb9d0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:19.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.282+0000 7fb9f1e00640 1 --2- 172.21.15.26:0/3231641203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9d00039b0 0x7fb9d0003dd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc009a90 tx=0x7fb9d0005830 comp rx=0 tx=0).stop 2024-04-26T21:10:19.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.282+0000 7fb9f1400640 1 -- 172.21.15.26:0/3231641203 reap_dead start 2024-04-26T21:10:19.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.282+0000 7fb9f1e00640 1 -- 172.21.15.26:0/3231641203 shutdown_connections 2024-04-26T21:10:19.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.282+0000 7fb9f1e00640 1 -- 172.21.15.26:0/3231641203 >> 172.21.15.26:0/3231641203 conn(0x7fb9ec06c690 msgr2=0x7fb9ec06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:19.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.282+0000 7fb9f1e00640 1 -- 172.21.15.26:0/3231641203 shutdown_connections 2024-04-26T21:10:19.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.282+0000 7fb9f1e00640 1 -- 172.21.15.26:0/3231641203 wait complete. 2024-04-26T21:10:19.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.284+0000 7fb9f1e00640 1 Processor -- start 2024-04-26T21:10:19.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.284+0000 7fb9f1e00640 1 -- start start 2024-04-26T21:10:19.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.285+0000 7fb9f1e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9d00039b0 0x7fb9ec152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:19.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.285+0000 7fb9f1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9ec1fd370 0x7fb9ec164ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:19.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.285+0000 7fb9f1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9ec165220 0x7fb9ec1f9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:19.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.285+0000 7fb9f1e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ec072da0 con 0x7fb9ec165220 2024-04-26T21:10:19.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.285+0000 7fb9f1e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ec167280 con 0x7fb9ec1fd370 2024-04-26T21:10:19.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.285+0000 7fb9f1e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb9ec06e3c0 con 0x7fb9d00039b0 2024-04-26T21:10:19.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.286+0000 7fb9f1400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9ec165220 0x7fb9ec1f9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:19.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.286+0000 7fb9f1400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9ec165220 0x7fb9ec1f9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57098/0 (socket says 172.21.15.26:57098) 2024-04-26T21:10:19.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.286+0000 7fb9f1400640 1 -- 172.21.15.26:0/1789037001 learned_addr learned my addr 172.21.15.26:0/1789037001 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:19.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.286+0000 7fb9ebe00640 1 --2- 172.21.15.26:0/1789037001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9ec1fd370 0x7fb9ec164ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:19.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.286+0000 7fb9f0a00640 1 --2- 172.21.15.26:0/1789037001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9d00039b0 0x7fb9ec152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:19.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.286+0000 7fb9f1400640 1 -- 172.21.15.26:0/1789037001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9d00039b0 msgr2=0x7fb9ec152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:19.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.286+0000 7fb9f1400640 1 --2- 172.21.15.26:0/1789037001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9d00039b0 0x7fb9ec152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:19.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.286+0000 7fb9f1400640 1 -- 172.21.15.26:0/1789037001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9ec1fd370 msgr2=0x7fb9ec164ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:19.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.286+0000 7fb9f1400640 1 --2- 172.21.15.26:0/1789037001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9ec1fd370 0x7fb9ec164ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:19.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.286+0000 7fb9f1400640 1 -- 172.21.15.26:0/1789037001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9ec1f9dc0 con 0x7fb9ec165220 2024-04-26T21:10:19.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.286+0000 7fb9f1400640 1 --2- 172.21.15.26:0/1789037001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9ec165220 0x7fb9ec1f9110 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0b5410 tx=0x7fb9e400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:19.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.287+0000 7fb9e8c00640 1 -- 172.21.15.26:0/1789037001 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e401c070 con 0x7fb9ec165220 2024-04-26T21:10:19.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.287+0000 7fb9e8c00640 1 -- 172.21.15.26:0/1789037001 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9e4017400 con 0x7fb9ec165220 2024-04-26T21:10:19.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.287+0000 7fb9e8c00640 1 -- 172.21.15.26:0/1789037001 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e40177f0 con 0x7fb9ec165220 2024-04-26T21:10:19.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.287+0000 7fb9f1e00640 1 -- 172.21.15.26:0/1789037001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9ec1fc890 con 0x7fb9ec165220 2024-04-26T21:10:19.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.287+0000 7fb9f1e00640 1 -- 172.21.15.26:0/1789037001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9ec1fce50 con 0x7fb9ec165220 2024-04-26T21:10:19.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.288+0000 7fb9f1e00640 1 -- 172.21.15.26:0/1789037001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9ec1fd050 con 0x7fb9ec165220 2024-04-26T21:10:19.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.289+0000 7fb9e8c00640 1 -- 172.21.15.26:0/1789037001 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9e401f070 con 0x7fb9ec165220 2024-04-26T21:10:19.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.291+0000 7fb9e8c00640 1 --2- 172.21.15.26:0/1789037001 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb9cc06c8f0 0x7fb9cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:19.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.291+0000 7fb9e8c00640 1 -- 172.21.15.26:0/1789037001 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7fb9e408bd00 con 0x7fb9ec165220 2024-04-26T21:10:19.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.291+0000 7fb9f0a00640 1 --2- 172.21.15.26:0/1789037001 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb9cc06c8f0 0x7fb9cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:19.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.292+0000 7fb9f0a00640 1 --2- 172.21.15.26:0/1789037001 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb9cc06c8f0 0x7fb9cc06edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc033000 tx=0x7fb9dc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:19.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.295+0000 7fb9e8c00640 1 -- 172.21.15.26:0/1789037001 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb9ec1fd050 con 0x7fb9ec165220 2024-04-26T21:10:19.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.500+0000 7fb9f1e00640 1 -- 172.21.15.26:0/1789037001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9ec1633b0 con 0x7fb9ec165220 2024-04-26T21:10:19.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.501+0000 7fb9e8c00640 1 -- 172.21.15.26:0/1789037001 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+18433 (secure 0 0 0) 0x7fb9e4012070 con 0x7fb9ec165220 2024-04-26T21:10:19.503 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:19.504 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":37,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:10:06.822858+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:09:46.646256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T21:09:47.613883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-04-26T21:10:04.249730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:10:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.504+0000 7fb9f1e00640 1 -- 172.21.15.26:0/1789037001 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb9cc06c8f0 msgr2=0x7fb9cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.504+0000 7fb9f1e00640 1 --2- 172.21.15.26:0/1789037001 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb9cc06c8f0 0x7fb9cc06edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc033000 tx=0x7fb9dc008030 comp rx=0 tx=0).stop 2024-04-26T21:10:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.504+0000 7fb9f1e00640 1 -- 172.21.15.26:0/1789037001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9ec165220 msgr2=0x7fb9ec1f9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.504+0000 7fb9f1e00640 1 --2- 172.21.15.26:0/1789037001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9ec165220 0x7fb9ec1f9110 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0b5410 tx=0x7fb9e400f4a0 comp rx=0 tx=0).stop 2024-04-26T21:10:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.504+0000 7fb9f1e00640 1 -- 172.21.15.26:0/1789037001 shutdown_connections 2024-04-26T21:10:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.504+0000 7fb9f1e00640 1 --2- 172.21.15.26:0/1789037001 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb9cc06c8f0 0x7fb9cc06edb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.504+0000 7fb9f1e00640 1 --2- 172.21.15.26:0/1789037001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9ec165220 0x7fb9ec1f9110 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.504+0000 7fb9f1e00640 1 --2- 172.21.15.26:0/1789037001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9ec1fd370 0x7fb9ec164ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.504+0000 7fb9f1e00640 1 --2- 172.21.15.26:0/1789037001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9d00039b0 0x7fb9ec152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.504+0000 7fb9f1e00640 1 -- 172.21.15.26:0/1789037001 >> 172.21.15.26:0/1789037001 conn(0x7fb9ec06c690 msgr2=0x7fb9ec156230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.504+0000 7fb9f1e00640 1 -- 172.21.15.26:0/1789037001 shutdown_connections 2024-04-26T21:10:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.505+0000 7fb9f1e00640 1 -- 172.21.15.26:0/1789037001 wait complete. 2024-04-26T21:10:19.526 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:10:19.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.738+0000 7f2410c00640 1 Processor -- start 2024-04-26T21:10:19.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.738+0000 7f2410c00640 1 -- start start 2024-04-26T21:10:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.740+0000 7f2410c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f240c146990 0x7f240c0b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.740+0000 7f2410c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f240c06ec20 0x7f240c0d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.740+0000 7f2410c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f240c0d1620 con 0x7f240c0d1210 2024-04-26T21:10:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.740+0000 7f2410c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f240c146d70 con 0x7f240c06ec20 2024-04-26T21:10:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.740+0000 7f2410c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f240c0b4480 con 0x7f240c146990 2024-04-26T21:10:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.740+0000 7f2403e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f240c06ec20 0x7f240c0d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.740+0000 7f240a800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f240c0d1210 0x7f240c072100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54568/0 (socket says 172.21.15.26:54568) 2024-04-26T21:10:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.741+0000 7f240a800640 1 -- 172.21.15.26:0/3627243777 learned_addr learned my addr 172.21.15.26:0/3627243777 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.741+0000 7f2409e00640 1 --2- 172.21.15.26:0/3627243777 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f240c146990 0x7f240c0b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.741+0000 7f2409400640 1 -- 172.21.15.26:0/3627243777 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1641093963 0 0) 0x7f2404001490 con 0x7f240c0d1210 2024-04-26T21:10:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.741+0000 7f2409400640 1 -- 172.21.15.26:0/3627243777 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23f0003520 con 0x7f240c0d1210 2024-04-26T21:10:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.741+0000 7f2403e00640 1 -- 172.21.15.26:0/3627243777 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f240c146990 msgr2=0x7f240c0b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.741+0000 7f2403e00640 1 --2- 172.21.15.26:0/3627243777 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f240c146990 0x7f240c0b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:19.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.741+0000 7f2403e00640 1 -- 172.21.15.26:0/3627243777 >> v1:172.21.15.26:6789/0 conn(0x7f240c0d1210 legacy=0x7f240c072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.741+0000 7f2403e00640 1 -- 172.21.15.26:0/3627243777 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240c073b70 con 0x7f240c06ec20 2024-04-26T21:10:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.742+0000 7f2403e00640 1 --2- 172.21.15.26:0/3627243777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f240c06ec20 0x7f240c0d0cd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f23f40097a0 tx=0x7f23f402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74e4227087ce415 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.743+0000 7f2409400640 1 -- 172.21.15.26:0/3627243777 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4037070 con 0x7f240c06ec20 2024-04-26T21:10:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.743+0000 7f2409400640 1 -- 172.21.15.26:0/3627243777 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f4032440 con 0x7f240c06ec20 2024-04-26T21:10:19.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.743+0000 7f2409400640 1 -- 172.21.15.26:0/3627243777 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f40327f0 con 0x7f240c06ec20 2024-04-26T21:10:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.743+0000 7f2410c00640 1 -- 172.21.15.26:0/3627243777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f240c06ec20 msgr2=0x7f240c0d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.743+0000 7f2410c00640 1 --2- 172.21.15.26:0/3627243777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f240c06ec20 0x7f240c0d0cd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f23f40097a0 tx=0x7f23f402a180 comp rx=0 tx=0).stop 2024-04-26T21:10:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.743+0000 7f2410c00640 1 -- 172.21.15.26:0/3627243777 shutdown_connections 2024-04-26T21:10:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.743+0000 7f2410c00640 1 --2- 172.21.15.26:0/3627243777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f240c06ec20 0x7f240c0d0cd0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.743+0000 7f2410c00640 1 --2- 172.21.15.26:0/3627243777 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f240c146990 0x7f240c0b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.743+0000 7f2410c00640 1 -- 172.21.15.26:0/3627243777 >> 172.21.15.26:0/3627243777 conn(0x7f240c0cd840 msgr2=0x7f240c0cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.744+0000 7f2410c00640 1 -- 172.21.15.26:0/3627243777 shutdown_connections 2024-04-26T21:10:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.744+0000 7f2410c00640 1 -- 172.21.15.26:0/3627243777 wait complete. 2024-04-26T21:10:19.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.745+0000 7f2410c00640 1 Processor -- start 2024-04-26T21:10:19.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.745+0000 7f2410c00640 1 -- start start 2024-04-26T21:10:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.746+0000 7f2410c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f240c06ec20 0x7f240c1f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.746+0000 7f2410c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f240c0d1210 0x7f240c15c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.746+0000 7f2410c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f240c146990 0x7f240c1e3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.746+0000 7f2410c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f240c06e420 con 0x7f240c146990 2024-04-26T21:10:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.746+0000 7f2410c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f240c074130 con 0x7f240c06ec20 2024-04-26T21:10:19.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.746+0000 7f2410c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f240c06e700 con 0x7f240c0d1210 2024-04-26T21:10:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.747+0000 7f240a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f240c146990 0x7f240c1e3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.747+0000 7f240a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f240c146990 0x7f240c1e3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57102/0 (socket says 172.21.15.26:57102) 2024-04-26T21:10:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.747+0000 7f240a800640 1 -- 172.21.15.26:0/3196219549 learned_addr learned my addr 172.21.15.26:0/3196219549 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.747+0000 7f2409e00640 1 --2- 172.21.15.26:0/3196219549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f240c06ec20 0x7f240c1f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.747+0000 7f2403e00640 1 --2- 172.21.15.26:0/3196219549 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f240c0d1210 0x7f240c15c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.747+0000 7f240a800640 1 -- 172.21.15.26:0/3196219549 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f240c0d1210 msgr2=0x7f240c15c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.747+0000 7f240a800640 1 --2- 172.21.15.26:0/3196219549 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f240c0d1210 0x7f240c15c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.747+0000 7f240a800640 1 -- 172.21.15.26:0/3196219549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f240c06ec20 msgr2=0x7f240c1f2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.747+0000 7f240a800640 1 --2- 172.21.15.26:0/3196219549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f240c06ec20 0x7f240c1f2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.747+0000 7f240a800640 1 -- 172.21.15.26:0/3196219549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240c1e5220 con 0x7f240c146990 2024-04-26T21:10:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.747+0000 7f240a800640 1 --2- 172.21.15.26:0/3196219549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f240c146990 0x7f240c1e3b70 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f240400cb80 tx=0x7f240400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.748+0000 7f2402000640 1 -- 172.21.15.26:0/3196219549 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404017070 con 0x7f240c146990 2024-04-26T21:10:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.748+0000 7f2402000640 1 -- 172.21.15.26:0/3196219549 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f240400ad40 con 0x7f240c146990 2024-04-26T21:10:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.748+0000 7f2402000640 1 -- 172.21.15.26:0/3196219549 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404012620 con 0x7f240c146990 2024-04-26T21:10:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.748+0000 7f2410c00640 1 -- 172.21.15.26:0/3196219549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f240c1e5500 con 0x7f240c146990 2024-04-26T21:10:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.748+0000 7f2410c00640 1 -- 172.21.15.26:0/3196219549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f240c1e5ac0 con 0x7f240c146990 2024-04-26T21:10:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.749+0000 7f2410c00640 1 -- 172.21.15.26:0/3196219549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f240c06a700 con 0x7f240c146990 2024-04-26T21:10:19.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.750+0000 7f2402000640 1 -- 172.21.15.26:0/3196219549 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f240401a070 con 0x7f240c146990 2024-04-26T21:10:19.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.751+0000 7f2402000640 1 --2- 172.21.15.26:0/3196219549 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f23e806c8f0 0x7f23e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:19.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.751+0000 7f2402000640 1 -- 172.21.15.26:0/3196219549 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f240408bc10 con 0x7f240c146990 2024-04-26T21:10:19.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.752+0000 7f2409e00640 1 --2- 172.21.15.26:0/3196219549 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f23e806c8f0 0x7f23e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:19.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.752+0000 7f2409e00640 1 --2- 172.21.15.26:0/3196219549 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f23e806c8f0 0x7f23e806edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f23fc006fd0 tx=0x7f23fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:19.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:19.756+0000 7f2402000640 1 -- 172.21.15.26:0/3196219549 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f240c0d1620 con 0x7f240c146990 2024-04-26T21:10:20.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.004+0000 7f2410c00640 1 -- 172.21.15.26:0/3196219549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f240c06fd50 con 0x7f240c146990 2024-04-26T21:10:20.007 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-04-26T21:10:20.005+0000 7f322fa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:10:20.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.010+0000 7f2402000640 1 -- 172.21.15.26:0/3196219549 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7f2404059e10 con 0x7f240c146990 2024-04-26T21:10:20.012 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:10:20.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.014+0000 7f2410c00640 1 -- 172.21.15.26:0/3196219549 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f23e806c8f0 msgr2=0x7f23e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:20.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.015+0000 7f2410c00640 1 --2- 172.21.15.26:0/3196219549 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f23e806c8f0 0x7f23e806edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f23fc006fd0 tx=0x7f23fc008040 comp rx=0 tx=0).stop 2024-04-26T21:10:20.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.015+0000 7f2410c00640 1 -- 172.21.15.26:0/3196219549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f240c146990 msgr2=0x7f240c1e3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:20.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.015+0000 7f2410c00640 1 --2- 172.21.15.26:0/3196219549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f240c146990 0x7f240c1e3b70 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f240400cb80 tx=0x7f240400a430 comp rx=0 tx=0).stop 2024-04-26T21:10:20.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.015+0000 7f2410c00640 1 -- 172.21.15.26:0/3196219549 shutdown_connections 2024-04-26T21:10:20.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.015+0000 7f2410c00640 1 --2- 172.21.15.26:0/3196219549 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f23e806c8f0 0x7f23e806edb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:20.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.015+0000 7f2410c00640 1 --2- 172.21.15.26:0/3196219549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f240c146990 0x7f240c1e3b70 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:20.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.015+0000 7f2410c00640 1 --2- 172.21.15.26:0/3196219549 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f240c0d1210 0x7f240c15c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:20.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.015+0000 7f2410c00640 1 --2- 172.21.15.26:0/3196219549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f240c06ec20 0x7f240c1f2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.015+0000 7f2410c00640 1 -- 172.21.15.26:0/3196219549 >> 172.21.15.26:0/3196219549 conn(0x7f240c0cd840 msgr2=0x7f240c0b59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.016+0000 7f2410c00640 1 -- 172.21.15.26:0/3196219549 shutdown_connections 2024-04-26T21:10:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.016+0000 7f2410c00640 1 -- 172.21.15.26:0/3196219549 wait complete. 2024-04-26T21:10:20.041 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:10:20.115 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-04-26T21:10:20.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.234+0000 7f41ffe00640 1 Processor -- start 2024-04-26T21:10:20.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.234+0000 7f41ffe00640 1 -- start start 2024-04-26T21:10:20.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.235+0000 7f41ffe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42000d1ba0 0x7f42000d36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:20.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.235+0000 7f41ffe00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42000b2430 con 0x7f42000b3c40 2024-04-26T21:10:20.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.235+0000 7f41ffe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42000cfe60 con 0x7f420015d030 2024-04-26T21:10:20.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.235+0000 7f41ffe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f42000b27a0 con 0x7f42000d1ba0 2024-04-26T21:10:20.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.235+0000 7f41fe000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f42000b3c40 0x7f4200164cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54584/0 (socket says 172.21.15.26:54584) 2024-04-26T21:10:20.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.235+0000 7f41fe000640 1 -- 172.21.15.26:0/1261202671 learned_addr learned my addr 172.21.15.26:0/1261202671 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.236+0000 7f41ff400640 1 --2- 172.21.15.26:0/1261202671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42000d1ba0 0x7f42000d36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.236+0000 7f41fd600640 1 -- 172.21.15.26:0/1261202671 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1502959 0 0) 0x7f42000b2430 con 0x7f42000b3c40 2024-04-26T21:10:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.236+0000 7f41fd600640 1 -- 172.21.15.26:0/1261202671 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41e4003520 con 0x7f42000b3c40 2024-04-26T21:10:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.236+0000 7f41fd600640 1 -- 172.21.15.26:0/1261202671 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1688995767 0 0) 0x7f41e4003520 con 0x7f42000b3c40 2024-04-26T21:10:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.236+0000 7f41fd600640 1 -- 172.21.15.26:0/1261202671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42000d1ba0 msgr2=0x7f42000d36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.236+0000 7f41fd600640 1 --2- 172.21.15.26:0/1261202671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42000d1ba0 0x7f42000d36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.236+0000 7f41fd600640 1 -- 172.21.15.26:0/1261202671 >> v1:172.21.15.86:6789/0 conn(0x7f420015d030 legacy=0x7f42000d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.236+0000 7f41fd600640 1 -- 172.21.15.26:0/1261202671 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42000d0140 con 0x7f42000b3c40 2024-04-26T21:10:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.236+0000 7f41fd600640 1 -- 172.21.15.26:0/1261202671 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f41ec001670 con 0x7f42000b3c40 2024-04-26T21:10:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.236+0000 7f41fd600640 1 -- 172.21.15.26:0/1261202671 >> v1:172.21.15.26:6789/0 conn(0x7f42000b3c40 legacy=0x7f4200164cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.236+0000 7f41fd600640 1 --2- 172.21.15.26:0/1261202671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e40039e0 0x7f41e4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.237+0000 7f41fd600640 1 --2- 172.21.15.26:0/1261202671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41e4004800 0x7f41e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.237+0000 7f41fd600640 1 --2- 172.21.15.26:0/1261202671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42000d1ba0 0x7f41e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.237+0000 7f41fd600640 1 -- 172.21.15.26:0/1261202671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f42000b27a0 con 0x7f41e4004800 2024-04-26T21:10:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.237+0000 7f41fd600640 1 -- 172.21.15.26:0/1261202671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41e400ba70 con 0x7f41e40039e0 2024-04-26T21:10:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.237+0000 7f41fd600640 1 -- 172.21.15.26:0/1261202671 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f41e400bc70 con 0x7f42000d1ba0 2024-04-26T21:10:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.237+0000 7f41fe000640 1 --2- 172.21.15.26:0/1261202671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41e4004800 0x7f41e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:20.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.237+0000 7f41fea00640 1 --2- 172.21.15.26:0/1261202671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e40039e0 0x7f41e4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:20.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.237+0000 7f41ff400640 1 --2- 172.21.15.26:0/1261202671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42000d1ba0 0x7f41e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:20.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.237+0000 7f41fe000640 1 -- 172.21.15.26:0/1261202671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42000d1ba0 msgr2=0x7f41e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:20.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.237+0000 7f41fe000640 1 --2- 172.21.15.26:0/1261202671 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42000d1ba0 0x7f41e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:20.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.237+0000 7f41fe000640 1 -- 172.21.15.26:0/1261202671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e40039e0 msgr2=0x7f41e4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:20.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.237+0000 7f41fe000640 1 --2- 172.21.15.26:0/1261202671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41e40039e0 0x7f41e4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:20.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.237+0000 7f41fe000640 1 -- 172.21.15.26:0/1261202671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e4003520 con 0x7f41e4004800 2024-04-26T21:10:20.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.238+0000 7f41fe000640 1 --2- 172.21.15.26:0/1261202671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41e4004800 0x7f41e4004c80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f41ec001eb0 tx=0x7f41e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d8029fcfc95bc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:20.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.239+0000 7f41fd600640 1 -- 172.21.15.26:0/1261202671 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec007d30 con 0x7f41e4004800 2024-04-26T21:10:20.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.239+0000 7f41fd600640 1 -- 172.21.15.26:0/1261202671 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41ec005af0 con 0x7f41e4004800 2024-04-26T21:10:20.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.239+0000 7f41fd600640 1 -- 172.21.15.26:0/1261202671 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec007070 con 0x7f41e4004800 2024-04-26T21:10:20.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.239+0000 7f41ffe00640 1 -- 172.21.15.26:0/1261202671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41e4004800 msgr2=0x7f41e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:20.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.239+0000 7f41ffe00640 1 --2- 172.21.15.26:0/1261202671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41e4004800 0x7f41e4004c80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f41ec001eb0 tx=0x7f41e4005830 comp rx=0 tx=0).stop 2024-04-26T21:10:20.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.239+0000 7f41ff400640 1 -- 172.21.15.26:0/1261202671 reap_dead start 2024-04-26T21:10:20.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.240+0000 7f41ffe00640 1 -- 172.21.15.26:0/1261202671 shutdown_connections 2024-04-26T21:10:20.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.240+0000 7f41ffe00640 1 -- 172.21.15.26:0/1261202671 >> 172.21.15.26:0/1261202671 conn(0x7f420009b9e0 msgr2=0x7f420009bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:20.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.240+0000 7f41ffe00640 1 -- 172.21.15.26:0/1261202671 shutdown_connections 2024-04-26T21:10:20.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.240+0000 7f41ffe00640 1 -- 172.21.15.26:0/1261202671 wait complete. 2024-04-26T21:10:20.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.241+0000 7f41ffe00640 1 Processor -- start 2024-04-26T21:10:20.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.242+0000 7f41ffe00640 1 -- start start 2024-04-26T21:10:20.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.242+0000 7f41ffe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41e4004800 0x7f42001524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:20.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.242+0000 7f41ffe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42001f8b70 0x7f42001f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:20.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.243+0000 7f41ffe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42001f84a0 0x7f42000b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:20.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.243+0000 7f41ffe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f420015d410 con 0x7f41e4004800 2024-04-26T21:10:20.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.243+0000 7f41ffe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42000d0700 con 0x7f42001f84a0 2024-04-26T21:10:20.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.243+0000 7f41ffe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f420015d590 con 0x7f42001f8b70 2024-04-26T21:10:20.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.243+0000 7f41fea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41e4004800 0x7f42001524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:20.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.243+0000 7f41fea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41e4004800 0x7f42001524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57134/0 (socket says 172.21.15.26:57134) 2024-04-26T21:10:20.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.243+0000 7f41fea00640 1 -- 172.21.15.26:0/3822487561 learned_addr learned my addr 172.21.15.26:0/3822487561 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:20.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.243+0000 7f41fe000640 1 --2- 172.21.15.26:0/3822487561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42001f8b70 0x7f42001f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:20.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.243+0000 7f41ff400640 1 --2- 172.21.15.26:0/3822487561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42001f84a0 0x7f42000b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:20.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.243+0000 7f41fea00640 1 -- 172.21.15.26:0/3822487561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42001f8b70 msgr2=0x7f42001f7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:20.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.243+0000 7f41fea00640 1 --2- 172.21.15.26:0/3822487561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42001f8b70 0x7f42001f7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:20.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.243+0000 7f41fea00640 1 -- 172.21.15.26:0/3822487561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42001f84a0 msgr2=0x7f42000b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:20.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.243+0000 7f41fea00640 1 --2- 172.21.15.26:0/3822487561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42001f84a0 0x7f42000b5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:20.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.243+0000 7f41fea00640 1 -- 172.21.15.26:0/3822487561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420006de00 con 0x7f41e4004800 2024-04-26T21:10:20.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.243+0000 7f41ff400640 1 --2- 172.21.15.26:0/3822487561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42001f84a0 0x7f42000b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:20.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.243+0000 7f41fe000640 1 --2- 172.21.15.26:0/3822487561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42001f8b70 0x7f42001f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:20.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.244+0000 7f41fea00640 1 --2- 172.21.15.26:0/3822487561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41e4004800 0x7f42001524f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f41f000e8f0 tx=0x7f41f000edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:20.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.244+0000 7f41f7400640 1 -- 172.21.15.26:0/3822487561 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0003b80 con 0x7f41e4004800 2024-04-26T21:10:20.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.244+0000 7f41f7400640 1 -- 172.21.15.26:0/3822487561 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f0018070 con 0x7f41e4004800 2024-04-26T21:10:20.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.244+0000 7f41f7400640 1 -- 172.21.15.26:0/3822487561 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f00050d0 con 0x7f41e4004800 2024-04-26T21:10:20.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.244+0000 7f41ffe00640 1 -- 172.21.15.26:0/3822487561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f420006e0e0 con 0x7f41e4004800 2024-04-26T21:10:20.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.244+0000 7f41ffe00640 1 -- 172.21.15.26:0/3822487561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f420006e6a0 con 0x7f41e4004800 2024-04-26T21:10:20.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.245+0000 7f41ffe00640 1 -- 172.21.15.26:0/3822487561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4200076250 con 0x7f41e4004800 2024-04-26T21:10:20.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.246+0000 7f41f7400640 1 -- 172.21.15.26:0/3822487561 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41f0005270 con 0x7f41e4004800 2024-04-26T21:10:20.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.248+0000 7f41f7400640 1 --2- 172.21.15.26:0/3822487561 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f41e006c8f0 0x7f41e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:20.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.248+0000 7f41f7400640 1 -- 172.21.15.26:0/3822487561 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6190+0+0 (secure 0 0 0) 0x7f41f008be80 con 0x7f41e4004800 2024-04-26T21:10:20.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.248+0000 7f41fe000640 1 --2- 172.21.15.26:0/3822487561 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f41e006c8f0 0x7f41e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:20.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.249+0000 7f41fe000640 1 --2- 172.21.15.26:0/3822487561 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f41e006c8f0 0x7f41e006edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f41ec032000 tx=0x7f41ec02d000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:20.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.253+0000 7f41f7400640 1 -- 172.21.15.26:0/3822487561 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f41f001a310 con 0x7f41e4004800 2024-04-26T21:10:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:20.498+0000 7f41ffe00640 1 -- 172.21.15.26:0/3822487561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f42001f88b0 con 0x7f41e4004800 2024-04-26T21:10:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.013+0000 7f41f7400640 1 -- 172.21.15.26:0/3822487561 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7f42001f88b0 con 0x7f41e4004800 2024-04-26T21:10:21.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.016+0000 7f41ffe00640 1 -- 172.21.15.26:0/3822487561 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f41e006c8f0 msgr2=0x7f41e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:21.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.016+0000 7f41ffe00640 1 --2- 172.21.15.26:0/3822487561 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f41e006c8f0 0x7f41e006edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f41ec032000 tx=0x7f41ec02d000 comp rx=0 tx=0).stop 2024-04-26T21:10:21.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.016+0000 7f41ffe00640 1 -- 172.21.15.26:0/3822487561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41e4004800 msgr2=0x7f42001524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:21.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.016+0000 7f41ffe00640 1 --2- 172.21.15.26:0/3822487561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41e4004800 0x7f42001524f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f41f000e8f0 tx=0x7f41f000edb0 comp rx=0 tx=0).stop 2024-04-26T21:10:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.017+0000 7f41ffe00640 1 -- 172.21.15.26:0/3822487561 shutdown_connections 2024-04-26T21:10:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.017+0000 7f41ffe00640 1 --2- 172.21.15.26:0/3822487561 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f41e006c8f0 0x7f41e006edb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.017+0000 7f41ffe00640 1 --2- 172.21.15.26:0/3822487561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42001f84a0 0x7f42000b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.017+0000 7f41ffe00640 1 --2- 172.21.15.26:0/3822487561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42001f8b70 0x7f42001f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.017+0000 7f41ffe00640 1 --2- 172.21.15.26:0/3822487561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41e4004800 0x7f42001524f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.017+0000 7f41ffe00640 1 -- 172.21.15.26:0/3822487561 >> 172.21.15.26:0/3822487561 conn(0x7f420009b9e0 msgr2=0x7f4200070fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.017+0000 7f41ffe00640 1 -- 172.21.15.26:0/3822487561 shutdown_connections 2024-04-26T21:10:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.017+0000 7f41ffe00640 1 -- 172.21.15.26:0/3822487561 wait complete. 2024-04-26T21:10:21.042 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:10:21.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.267+0000 7f091d200640 1 Processor -- start 2024-04-26T21:10:21.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.267+0000 7f091d200640 1 -- start start 2024-04-26T21:10:21.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.268+0000 7f091d200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0918148050 0x7f091806c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:21.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.269+0000 7f091d200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f091806c780 0x7f09180b5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:21.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.269+0000 7f091d200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09180733f0 con 0x7f0918147c70 2024-04-26T21:10:21.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.269+0000 7f091d200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f091806cbd0 con 0x7f091806c780 2024-04-26T21:10:21.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.269+0000 7f091d200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f09180736d0 con 0x7f0918148050 2024-04-26T21:10:21.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.269+0000 7f0917400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0918147c70 0x7f0918148500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54598/0 (socket says 172.21.15.26:54598) 2024-04-26T21:10:21.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.269+0000 7f0917400640 1 -- 172.21.15.26:0/867685273 learned_addr learned my addr 172.21.15.26:0/867685273 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:21.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.269+0000 7f0916a00640 1 --2- 172.21.15.26:0/867685273 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0918148050 0x7f091806c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:21.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.269+0000 7f0917e00640 1 --2- 172.21.15.26:0/867685273 >> v2:172.21.15.86:3300/0 conn(0x7f091806c780 0x7f09180b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:21.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.269+0000 7f0916000640 1 -- 172.21.15.26:0/867685273 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4247706096 0 0) 0x7f09180733f0 con 0x7f0918147c70 2024-04-26T21:10:21.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.270+0000 7f0916000640 1 -- 172.21.15.26:0/867685273 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08fc003520 con 0x7f0918147c70 2024-04-26T21:10:21.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.270+0000 7f0916000640 1 -- 172.21.15.26:0/867685273 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3403902508 0 0) 0x7f08fc003520 con 0x7f0918147c70 2024-04-26T21:10:21.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.270+0000 7f0916000640 1 -- 172.21.15.26:0/867685273 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0918148050 msgr2=0x7f091806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:21.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.270+0000 7f0916000640 1 --2- 172.21.15.26:0/867685273 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0918148050 0x7f091806c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:21.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.270+0000 7f0916000640 1 -- 172.21.15.26:0/867685273 >> v2:172.21.15.86:3300/0 conn(0x7f091806c780 msgr2=0x7f09180b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:21.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.270+0000 7f0916000640 1 --2- 172.21.15.26:0/867685273 >> v2:172.21.15.86:3300/0 conn(0x7f091806c780 0x7f09180b5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:21.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.270+0000 7f0916000640 1 -- 172.21.15.26:0/867685273 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09180739b0 con 0x7f0918147c70 2024-04-26T21:10:21.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.270+0000 7f0916000640 1 -- 172.21.15.26:0/867685273 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f09080019f0 con 0x7f0918147c70 2024-04-26T21:10:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.270+0000 7f0916000640 1 -- 172.21.15.26:0/867685273 >> v1:172.21.15.26:6789/0 conn(0x7f0918147c70 legacy=0x7f0918148500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.270+0000 7f0916000640 1 --2- 172.21.15.26:0/867685273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08fc0039b0 0x7f08fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.270+0000 7f0916000640 1 --2- 172.21.15.26:0/867685273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08fc004800 0x7f08fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.270+0000 7f0916000640 1 --2- 172.21.15.26:0/867685273 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0918148050 0x7f08fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.270+0000 7f0916000640 1 -- 172.21.15.26:0/867685273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f091806cbd0 con 0x7f08fc004800 2024-04-26T21:10:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.270+0000 7f0916000640 1 -- 172.21.15.26:0/867685273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f09180736d0 con 0x7f08fc0039b0 2024-04-26T21:10:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.270+0000 7f0916000640 1 -- 172.21.15.26:0/867685273 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f08fc00b9c0 con 0x7f0918148050 2024-04-26T21:10:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.270+0000 7f0916a00640 1 --2- 172.21.15.26:0/867685273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08fc004800 0x7f08fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.271+0000 7f0916a00640 1 -- 172.21.15.26:0/867685273 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0918148050 msgr2=0x7f08fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.271+0000 7f0916a00640 1 --2- 172.21.15.26:0/867685273 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0918148050 0x7f08fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.271+0000 7f0916a00640 1 -- 172.21.15.26:0/867685273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08fc0039b0 msgr2=0x7f08fc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.271+0000 7f0917400640 1 --2- 172.21.15.26:0/867685273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08fc0039b0 0x7f08fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:21.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.271+0000 7f0916a00640 1 --2- 172.21.15.26:0/867685273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08fc0039b0 0x7f08fc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:21.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.271+0000 7f0916a00640 1 -- 172.21.15.26:0/867685273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08fc003520 con 0x7f08fc004800 2024-04-26T21:10:21.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.271+0000 7f0917400640 1 --2- 172.21.15.26:0/867685273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08fc0039b0 0x7f08fc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:21.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.272+0000 7f0916a00640 1 --2- 172.21.15.26:0/867685273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08fc004800 0x7f08fc004c80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0904009ac0 tx=0x7f08fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6c2df628cc85aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:21.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.272+0000 7f0916000640 1 -- 172.21.15.26:0/867685273 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904036070 con 0x7f08fc004800 2024-04-26T21:10:21.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.272+0000 7f0916000640 1 -- 172.21.15.26:0/867685273 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0904031400 con 0x7f08fc004800 2024-04-26T21:10:21.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.272+0000 7f0916000640 1 -- 172.21.15.26:0/867685273 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904009690 con 0x7f08fc004800 2024-04-26T21:10:21.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.273+0000 7f091d200640 1 -- 172.21.15.26:0/867685273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08fc004800 msgr2=0x7f08fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:21.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.273+0000 7f091d200640 1 --2- 172.21.15.26:0/867685273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08fc004800 0x7f08fc004c80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0904009ac0 tx=0x7f08fc005830 comp rx=0 tx=0).stop 2024-04-26T21:10:21.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.273+0000 7f0917e00640 1 -- 172.21.15.26:0/867685273 reap_dead start 2024-04-26T21:10:21.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.273+0000 7f091d200640 1 -- 172.21.15.26:0/867685273 shutdown_connections 2024-04-26T21:10:21.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.273+0000 7f091d200640 1 -- 172.21.15.26:0/867685273 >> 172.21.15.26:0/867685273 conn(0x7f091806de40 msgr2=0x7f09180b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:21.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.273+0000 7f091d200640 1 -- 172.21.15.26:0/867685273 shutdown_connections 2024-04-26T21:10:21.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.273+0000 7f091d200640 1 -- 172.21.15.26:0/867685273 wait complete. 2024-04-26T21:10:21.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.275+0000 7f091d200640 1 Processor -- start 2024-04-26T21:10:21.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.275+0000 7f091d200640 1 -- start start 2024-04-26T21:10:21.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.276+0000 7f091d200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08fc004800 0x7f091815f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:21.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.276+0000 7f091d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09180c6d70 0x7f09180c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:21.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.276+0000 7f091d200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09180b0cd0 0x7f09180b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:21.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.276+0000 7f091d200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0918073080 con 0x7f08fc004800 2024-04-26T21:10:21.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.276+0000 7f091d200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0918073f70 con 0x7f09180c6d70 2024-04-26T21:10:21.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.276+0000 7f091d200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0918148c10 con 0x7f09180b0cd0 2024-04-26T21:10:21.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.276+0000 7f0917400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08fc004800 0x7f091815f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:21.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.276+0000 7f0917400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08fc004800 0x7f091815f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57150/0 (socket says 172.21.15.26:57150) 2024-04-26T21:10:21.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.276+0000 7f0917400640 1 -- 172.21.15.26:0/958166786 learned_addr learned my addr 172.21.15.26:0/958166786 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:21.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.277+0000 7f0917e00640 1 --2- 172.21.15.26:0/958166786 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09180b0cd0 0x7f09180b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:21.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.277+0000 7f0917400640 1 -- 172.21.15.26:0/958166786 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09180b0cd0 msgr2=0x7f09180b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:21.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.277+0000 7f0917400640 1 --2- 172.21.15.26:0/958166786 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09180b0cd0 0x7f09180b1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:21.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.277+0000 7f0917400640 1 -- 172.21.15.26:0/958166786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09180c6d70 msgr2=0x7f09180c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:21.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.277+0000 7f0917400640 1 --2- 172.21.15.26:0/958166786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09180c6d70 0x7f09180c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:21.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.277+0000 7f0917400640 1 -- 172.21.15.26:0/958166786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09180c7810 con 0x7f08fc004800 2024-04-26T21:10:21.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.277+0000 7f0917400640 1 --2- 172.21.15.26:0/958166786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08fc004800 0x7f091815f8b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0908001b00 tx=0x7f090800b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:21.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.277+0000 7f090f400640 1 -- 172.21.15.26:0/958166786 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908019070 con 0x7f08fc004800 2024-04-26T21:10:21.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.277+0000 7f090f400640 1 -- 172.21.15.26:0/958166786 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0908004d60 con 0x7f08fc004800 2024-04-26T21:10:21.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.277+0000 7f091d200640 1 -- 172.21.15.26:0/958166786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09180c7af0 con 0x7f08fc004800 2024-04-26T21:10:21.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.277+0000 7f090f400640 1 -- 172.21.15.26:0/958166786 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09080072f0 con 0x7f08fc004800 2024-04-26T21:10:21.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.277+0000 7f091d200640 1 -- 172.21.15.26:0/958166786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09180aea60 con 0x7f08fc004800 2024-04-26T21:10:21.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.279+0000 7f091d200640 1 -- 172.21.15.26:0/958166786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09180aec60 con 0x7f08fc004800 2024-04-26T21:10:21.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.280+0000 7f090f400640 1 -- 172.21.15.26:0/958166786 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f090801c070 con 0x7f08fc004800 2024-04-26T21:10:21.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.281+0000 7f090f400640 1 --2- 172.21.15.26:0/958166786 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f08f806c8f0 0x7f08f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:21.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.281+0000 7f090f400640 1 -- 172.21.15.26:0/958166786 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6190+0+0 (secure 0 0 0) 0x7f090808beb0 con 0x7f08fc004800 2024-04-26T21:10:21.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.282+0000 7f0916a00640 1 --2- 172.21.15.26:0/958166786 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f08f806c8f0 0x7f08f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:21.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.282+0000 7f0916a00640 1 --2- 172.21.15.26:0/958166786 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f08f806c8f0 0x7f08f806edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0904033000 tx=0x7f090402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:21.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.286+0000 7f090f400640 1 -- 172.21.15.26:0/958166786 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f09180aec60 con 0x7f08fc004800 2024-04-26T21:10:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:21.491+0000 7f091d200640 1 -- 172.21.15.26:0/958166786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f09180ae850 con 0x7f08fc004800 2024-04-26T21:10:22.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.025+0000 7f090f400640 1 -- 172.21.15.26:0/958166786 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v39) ==== 181+0+0 (secure 0 0 0) 0x7f090805a190 con 0x7f08fc004800 2024-04-26T21:10:22.027 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:10:22.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.029+0000 7f091d200640 1 -- 172.21.15.26:0/958166786 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f08f806c8f0 msgr2=0x7f08f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:22.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.029+0000 7f091d200640 1 --2- 172.21.15.26:0/958166786 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f08f806c8f0 0x7f08f806edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0904033000 tx=0x7f090402e000 comp rx=0 tx=0).stop 2024-04-26T21:10:22.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.029+0000 7f091d200640 1 -- 172.21.15.26:0/958166786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08fc004800 msgr2=0x7f091815f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:22.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.029+0000 7f091d200640 1 --2- 172.21.15.26:0/958166786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08fc004800 0x7f091815f8b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0908001b00 tx=0x7f090800b560 comp rx=0 tx=0).stop 2024-04-26T21:10:22.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.029+0000 7f091d200640 1 -- 172.21.15.26:0/958166786 shutdown_connections 2024-04-26T21:10:22.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.029+0000 7f091d200640 1 --2- 172.21.15.26:0/958166786 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f08f806c8f0 0x7f08f806edb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:22.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.029+0000 7f091d200640 1 --2- 172.21.15.26:0/958166786 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09180b0cd0 0x7f09180b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:22.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.029+0000 7f091d200640 1 --2- 172.21.15.26:0/958166786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09180c6d70 0x7f09180c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.029+0000 7f091d200640 1 --2- 172.21.15.26:0/958166786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08fc004800 0x7f091815f8b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.029+0000 7f091d200640 1 -- 172.21.15.26:0/958166786 >> 172.21.15.26:0/958166786 conn(0x7f091806de40 msgr2=0x7f09180b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.030+0000 7f091d200640 1 -- 172.21.15.26:0/958166786 shutdown_connections 2024-04-26T21:10:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.030+0000 7f091d200640 1 -- 172.21.15.26:0/958166786 wait complete. 2024-04-26T21:10:22.060 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:10:22.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.262+0000 7f518f200640 1 Processor -- start 2024-04-26T21:10:22.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.263+0000 7f518f200640 1 -- start start 2024-04-26T21:10:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.264+0000 7f518f200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5190143790 0x7f51900ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.264+0000 7f518f200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5190146460 0x7f519006f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.264+0000 7f518f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f519006f900 con 0x7f5190146a90 2024-04-26T21:10:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.264+0000 7f518f200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f5190070410 con 0x7f5190143790 2024-04-26T21:10:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.264+0000 7f518f200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f51900b41e0 con 0x7f5190146460 2024-04-26T21:10:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.265+0000 7f518de00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5190143790 0x7f51900ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.265+0000 7f518de00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5190143790 0x7f51900ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57152/0 (socket says 172.21.15.26:57152) 2024-04-26T21:10:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.265+0000 7f518de00640 1 -- 172.21.15.26:0/1438139042 learned_addr learned my addr 172.21.15.26:0/1438139042 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.265+0000 7f518e800640 1 --2- 172.21.15.26:0/1438139042 >> v2:172.21.15.86:3300/0 conn(0x7f5190146460 0x7f519006f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.265+0000 7f518de00640 1 -- 172.21.15.26:0/1438139042 >> v2:172.21.15.86:3300/0 conn(0x7f5190146460 msgr2=0x7f519006f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.265+0000 7f518de00640 1 --2- 172.21.15.26:0/1438139042 >> v2:172.21.15.86:3300/0 conn(0x7f5190146460 0x7f519006f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:22.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.265+0000 7f518de00640 1 -- 172.21.15.26:0/1438139042 >> v1:172.21.15.86:6789/0 conn(0x7f5190146a90 legacy=0x7f519006ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:22.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.265+0000 7f518de00640 1 -- 172.21.15.26:0/1438139042 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51900b3eb0 con 0x7f5190143790 2024-04-26T21:10:22.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.266+0000 7f518de00640 1 --2- 172.21.15.26:0/1438139042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5190143790 0x7f51900ceb60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f51900ce990 tx=0x7f518002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a92c237e804907 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.267+0000 7f518d400640 1 -- 172.21.15.26:0/1438139042 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180037070 con 0x7f5190143790 2024-04-26T21:10:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.267+0000 7f518d400640 1 -- 172.21.15.26:0/1438139042 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5180032440 con 0x7f5190143790 2024-04-26T21:10:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.267+0000 7f518d400640 1 -- 172.21.15.26:0/1438139042 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180032810 con 0x7f5190143790 2024-04-26T21:10:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.267+0000 7f518f200640 1 -- 172.21.15.26:0/1438139042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5190143790 msgr2=0x7f51900ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.267+0000 7f518f200640 1 --2- 172.21.15.26:0/1438139042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5190143790 0x7f51900ceb60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f51900ce990 tx=0x7f518002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:10:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.268+0000 7f518f200640 1 -- 172.21.15.26:0/1438139042 shutdown_connections 2024-04-26T21:10:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.268+0000 7f518f200640 1 --2- 172.21.15.26:0/1438139042 >> v2:172.21.15.86:3300/0 conn(0x7f5190146460 0x7f519006f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.268+0000 7f518f200640 1 --2- 172.21.15.26:0/1438139042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5190143790 0x7f51900ceb60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.268+0000 7f518f200640 1 -- 172.21.15.26:0/1438139042 >> 172.21.15.26:0/1438139042 conn(0x7f51900b2a10 msgr2=0x7f51900b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.268+0000 7f518f200640 1 -- 172.21.15.26:0/1438139042 shutdown_connections 2024-04-26T21:10:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.268+0000 7f518f200640 1 -- 172.21.15.26:0/1438139042 wait complete. 2024-04-26T21:10:22.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.269+0000 7f518f200640 1 Processor -- start 2024-04-26T21:10:22.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.270+0000 7f518f200640 1 -- start start 2024-04-26T21:10:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.271+0000 7f518f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5190143790 0x7f51900af260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.271+0000 7f518f200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5190146460 0x7f519015d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.271+0000 7f518f200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5190146a90 0x7f5190145070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.271+0000 7f518f200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5190160d30 con 0x7f5190146460 2024-04-26T21:10:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.271+0000 7f518f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5190070130 con 0x7f5190143790 2024-04-26T21:10:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.271+0000 7f518f200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f5190160eb0 con 0x7f5190146a90 2024-04-26T21:10:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.271+0000 7f5187e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5190146460 0x7f519015d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.271+0000 7f5187e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5190146460 0x7f519015d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57156/0 (socket says 172.21.15.26:57156) 2024-04-26T21:10:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.271+0000 7f5187e00640 1 -- 172.21.15.26:0/4085435537 learned_addr learned my addr 172.21.15.26:0/4085435537 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:22.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.271+0000 7f518de00640 1 --2- 172.21.15.26:0/4085435537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5190143790 0x7f51900af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:22.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.271+0000 7f5187e00640 1 -- 172.21.15.26:0/4085435537 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5190146a90 msgr2=0x7f5190145070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:22.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.272+0000 7f518e800640 1 --2- 172.21.15.26:0/4085435537 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5190146a90 0x7f5190145070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:22.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.272+0000 7f5187e00640 1 --2- 172.21.15.26:0/4085435537 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5190146a90 0x7f5190145070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:22.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.272+0000 7f5187e00640 1 -- 172.21.15.26:0/4085435537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5190143790 msgr2=0x7f51900af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:22.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.272+0000 7f5187e00640 1 --2- 172.21.15.26:0/4085435537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5190143790 0x7f51900af260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:22.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.272+0000 7f5187e00640 1 -- 172.21.15.26:0/4085435537 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51900af7a0 con 0x7f5190146460 2024-04-26T21:10:22.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.272+0000 7f5187e00640 1 --2- 172.21.15.26:0/4085435537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5190146460 0x7f519015d5b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f517800b3f0 tx=0x7f517800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:22.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.272+0000 7f5186000640 1 -- 172.21.15.26:0/4085435537 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178015020 con 0x7f5190146460 2024-04-26T21:10:22.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.272+0000 7f5186000640 1 -- 172.21.15.26:0/4085435537 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5178011830 con 0x7f5190146460 2024-04-26T21:10:22.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.272+0000 7f5186000640 1 -- 172.21.15.26:0/4085435537 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178011c20 con 0x7f5190146460 2024-04-26T21:10:22.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.272+0000 7f518f200640 1 -- 172.21.15.26:0/4085435537 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51900afd20 con 0x7f5190146460 2024-04-26T21:10:22.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.272+0000 7f518f200640 1 -- 172.21.15.26:0/4085435537 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51900b02e0 con 0x7f5190146460 2024-04-26T21:10:22.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.273+0000 7f518f200640 1 -- 172.21.15.26:0/4085435537 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f519006a700 con 0x7f5190146460 2024-04-26T21:10:22.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.275+0000 7f5186000640 1 -- 172.21.15.26:0/4085435537 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5178018020 con 0x7f5190146460 2024-04-26T21:10:22.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.277+0000 7f5186000640 1 --2- 172.21.15.26:0/4085435537 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f516c06c8f0 0x7f516c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:22.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.277+0000 7f5186000640 1 -- 172.21.15.26:0/4085435537 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5752+0+0 (secure 0 0 0) 0x7f517808bae0 con 0x7f5190146460 2024-04-26T21:10:22.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.277+0000 7f518de00640 1 --2- 172.21.15.26:0/4085435537 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f516c06c8f0 0x7f516c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:22.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.278+0000 7f518de00640 1 --2- 172.21.15.26:0/4085435537 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f516c06c8f0 0x7f516c06edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5180034000 tx=0x7f518002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:22.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.281+0000 7f5186000640 1 -- 172.21.15.26:0/4085435537 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f517801f020 con 0x7f5190146460 2024-04-26T21:10:22.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:22.494+0000 7f518f200640 1 -- 172.21.15.26:0/4085435537 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f51900b04e0 con 0x7f5190146460 2024-04-26T21:10:23.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.037+0000 7f5186000640 1 -- 172.21.15.26:0/4085435537 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v40) ==== 169+0+0 (secure 0 0 0) 0x7f51780172c0 con 0x7f5190146460 2024-04-26T21:10:23.039 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-04-26T21:10:23.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.040+0000 7f518f200640 1 -- 172.21.15.26:0/4085435537 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f516c06c8f0 msgr2=0x7f516c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:23.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.040+0000 7f518f200640 1 --2- 172.21.15.26:0/4085435537 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f516c06c8f0 0x7f516c06edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5180034000 tx=0x7f518002e000 comp rx=0 tx=0).stop 2024-04-26T21:10:23.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.040+0000 7f518f200640 1 -- 172.21.15.26:0/4085435537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5190146460 msgr2=0x7f519015d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:23.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.040+0000 7f518f200640 1 --2- 172.21.15.26:0/4085435537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5190146460 0x7f519015d5b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f517800b3f0 tx=0x7f517800b8b0 comp rx=0 tx=0).stop 2024-04-26T21:10:23.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.040+0000 7f518f200640 1 -- 172.21.15.26:0/4085435537 shutdown_connections 2024-04-26T21:10:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.040+0000 7f518f200640 1 --2- 172.21.15.26:0/4085435537 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f516c06c8f0 0x7f516c06edb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.040+0000 7f518f200640 1 --2- 172.21.15.26:0/4085435537 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5190146a90 0x7f5190145070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.040+0000 7f518f200640 1 --2- 172.21.15.26:0/4085435537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5190146460 0x7f519015d5b0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.040+0000 7f518f200640 1 --2- 172.21.15.26:0/4085435537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5190143790 0x7f51900af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.040+0000 7f518f200640 1 -- 172.21.15.26:0/4085435537 >> 172.21.15.26:0/4085435537 conn(0x7f51900b2a10 msgr2=0x7f51900b0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.041+0000 7f518f200640 1 -- 172.21.15.26:0/4085435537 shutdown_connections 2024-04-26T21:10:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.041+0000 7f518f200640 1 -- 172.21.15.26:0/4085435537 wait complete. 2024-04-26T21:10:23.074 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm data_new data_new --yes-i-really-really-mean-it 2024-04-26T21:10:23.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.272+0000 7f050d200640 1 Processor -- start 2024-04-26T21:10:23.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.273+0000 7f050d200640 1 -- start start 2024-04-26T21:10:23.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.274+0000 7f050d200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f050806c890 0x7f0508159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.274+0000 7f050d200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05081606c0 con 0x7f050806c4b0 2024-04-26T21:10:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.274+0000 7f050d200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05080d1d10 con 0x7f05080d27d0 2024-04-26T21:10:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.274+0000 7f050d200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f05080d2c60 con 0x7f050806c890 2024-04-26T21:10:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.274+0000 7f0505e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f050806c4b0 0x7f05081593b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54602/0 (socket says 172.21.15.26:54602) 2024-04-26T21:10:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.274+0000 7f0505e00640 1 -- 172.21.15.26:0/3963022358 learned_addr learned my addr 172.21.15.26:0/3963022358 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.275+0000 7f0505400640 1 --2- 172.21.15.26:0/3963022358 >> v2:172.21.15.86:3300/0 conn(0x7f050806c890 0x7f0508159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.275+0000 7f0504a00640 1 -- 172.21.15.26:0/3963022358 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 101015315 0 0) 0x7f05081606c0 con 0x7f050806c4b0 2024-04-26T21:10:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.275+0000 7f0504a00640 1 -- 172.21.15.26:0/3963022358 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04ec003520 con 0x7f050806c4b0 2024-04-26T21:10:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.275+0000 7f0504a00640 1 -- 172.21.15.26:0/3963022358 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2003921822 0 0) 0x7f04ec003520 con 0x7f050806c4b0 2024-04-26T21:10:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.275+0000 7f0504a00640 1 -- 172.21.15.26:0/3963022358 >> v2:172.21.15.86:3300/0 conn(0x7f050806c890 msgr2=0x7f0508159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.275+0000 7f0504a00640 1 --2- 172.21.15.26:0/3963022358 >> v2:172.21.15.86:3300/0 conn(0x7f050806c890 0x7f0508159ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.275+0000 7f0504a00640 1 -- 172.21.15.26:0/3963022358 >> v1:172.21.15.86:6789/0 conn(0x7f05080d27d0 legacy=0x7f050815f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.275+0000 7f0504a00640 1 -- 172.21.15.26:0/3963022358 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05080d1ff0 con 0x7f050806c4b0 2024-04-26T21:10:23.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.276+0000 7f0504a00640 1 -- 172.21.15.26:0/3963022358 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f04f80019f0 con 0x7f050806c4b0 2024-04-26T21:10:23.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.276+0000 7f0504a00640 1 -- 172.21.15.26:0/3963022358 >> v1:172.21.15.26:6789/0 conn(0x7f050806c4b0 legacy=0x7f05081593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:23.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.276+0000 7f0504a00640 1 --2- 172.21.15.26:0/3963022358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04ec0039b0 0x7f04ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:23.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.276+0000 7f0504a00640 1 --2- 172.21.15.26:0/3963022358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04ec004800 0x7f04ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:23.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.276+0000 7f0504a00640 1 --2- 172.21.15.26:0/3963022358 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04ec0051c0 0x7f04ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:23.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.276+0000 7f0504a00640 1 -- 172.21.15.26:0/3963022358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f05080d2c60 con 0x7f04ec0039b0 2024-04-26T21:10:23.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.276+0000 7f0504a00640 1 -- 172.21.15.26:0/3963022358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04ec00c060 con 0x7f04ec004800 2024-04-26T21:10:23.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.276+0000 7f0504a00640 1 -- 172.21.15.26:0/3963022358 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f04ec00c260 con 0x7f04ec0051c0 2024-04-26T21:10:23.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.276+0000 7f0505e00640 1 --2- 172.21.15.26:0/3963022358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04ec0039b0 0x7f04ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:23.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.276+0000 7f0505e00640 1 -- 172.21.15.26:0/3963022358 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04ec0051c0 msgr2=0x7f04ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:23.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.276+0000 7f0505e00640 1 --2- 172.21.15.26:0/3963022358 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04ec0051c0 0x7f04ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:23.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.276+0000 7f0505e00640 1 -- 172.21.15.26:0/3963022358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04ec004800 msgr2=0x7f04ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:23.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.276+0000 7f0505e00640 1 --2- 172.21.15.26:0/3963022358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04ec004800 0x7f04ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:23.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.276+0000 7f0505e00640 1 -- 172.21.15.26:0/3963022358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ec003520 con 0x7f04ec0039b0 2024-04-26T21:10:23.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.276+0000 7f0506800640 1 -- 172.21.15.26:0/3963022358 reap_dead start 2024-04-26T21:10:23.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.277+0000 7f0505e00640 1 --2- 172.21.15.26:0/3963022358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04ec0039b0 0x7f04ec003dd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f04f800a840 tx=0x7f04ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fb116e3e810c44 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:23.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.278+0000 7f0504a00640 1 -- 172.21.15.26:0/3963022358 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f800a460 con 0x7f04ec0039b0 2024-04-26T21:10:23.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.278+0000 7f0504a00640 1 -- 172.21.15.26:0/3963022358 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f802f070 con 0x7f04ec0039b0 2024-04-26T21:10:23.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.278+0000 7f0504a00640 1 -- 172.21.15.26:0/3963022358 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f802bd30 con 0x7f04ec0039b0 2024-04-26T21:10:23.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.278+0000 7f050d200640 1 -- 172.21.15.26:0/3963022358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04ec0039b0 msgr2=0x7f04ec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:23.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.278+0000 7f050d200640 1 --2- 172.21.15.26:0/3963022358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04ec0039b0 0x7f04ec003dd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f04f800a840 tx=0x7f04ec005830 comp rx=0 tx=0).stop 2024-04-26T21:10:23.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.278+0000 7f050d200640 1 -- 172.21.15.26:0/3963022358 shutdown_connections 2024-04-26T21:10:23.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.278+0000 7f050d200640 1 --2- 172.21.15.26:0/3963022358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04ec0039b0 0x7f04ec003dd0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:23.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.278+0000 7f050d200640 1 -- 172.21.15.26:0/3963022358 >> 172.21.15.26:0/3963022358 conn(0x7f050806d350 msgr2=0x7f050806d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:23.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.279+0000 7f050d200640 1 -- 172.21.15.26:0/3963022358 shutdown_connections 2024-04-26T21:10:23.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.279+0000 7f050d200640 1 -- 172.21.15.26:0/3963022358 wait complete. 2024-04-26T21:10:23.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.281+0000 7f050d200640 1 Processor -- start 2024-04-26T21:10:23.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.281+0000 7f050d200640 1 -- start start 2024-04-26T21:10:23.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.282+0000 7f050d200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04ec0039b0 0x7f05081431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:23.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.282+0000 7f050d200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05080af590 0x7f050813d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:23.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.282+0000 7f050d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05080b1170 0x7f05080b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:23.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.282+0000 7f050d200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0508159530 con 0x7f05080af590 2024-04-26T21:10:23.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.282+0000 7f050d200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05081593b0 con 0x7f05080b1170 2024-04-26T21:10:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.282+0000 7f050d200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f05081596b0 con 0x7f04ec0039b0 2024-04-26T21:10:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.282+0000 7f0505400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05080af590 0x7f050813d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.282+0000 7f0505400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05080af590 0x7f050813d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57172/0 (socket says 172.21.15.26:57172) 2024-04-26T21:10:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.282+0000 7f0505400640 1 -- 172.21.15.26:0/1369810539 learned_addr learned my addr 172.21.15.26:0/1369810539 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.282+0000 7f0505e00640 1 --2- 172.21.15.26:0/1369810539 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04ec0039b0 0x7f05081431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.282+0000 7f0506800640 1 --2- 172.21.15.26:0/1369810539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05080b1170 0x7f05080b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.282+0000 7f0505400640 1 -- 172.21.15.26:0/1369810539 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04ec0039b0 msgr2=0x7f05081431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.282+0000 7f0505400640 1 --2- 172.21.15.26:0/1369810539 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04ec0039b0 0x7f05081431f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.282+0000 7f0505400640 1 -- 172.21.15.26:0/1369810539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05080b1170 msgr2=0x7f05080b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.282+0000 7f0505400640 1 --2- 172.21.15.26:0/1369810539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05080b1170 0x7f05080b15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.283+0000 7f0505400640 1 -- 172.21.15.26:0/1369810539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05080afb00 con 0x7f05080af590 2024-04-26T21:10:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.283+0000 7f0506800640 1 --2- 172.21.15.26:0/1369810539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05080b1170 0x7f05080b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.283+0000 7f0505400640 1 --2- 172.21.15.26:0/1369810539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05080af590 0x7f050813d630 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f04f000e7d0 tx=0x7f04f000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:23.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.283+0000 7f04fe000640 1 -- 172.21.15.26:0/1369810539 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0017070 con 0x7f05080af590 2024-04-26T21:10:23.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.283+0000 7f04fe000640 1 -- 172.21.15.26:0/1369810539 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f000ee80 con 0x7f05080af590 2024-04-26T21:10:23.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.283+0000 7f04fe000640 1 -- 172.21.15.26:0/1369810539 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f000cb80 con 0x7f05080af590 2024-04-26T21:10:23.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.283+0000 7f050d200640 1 -- 172.21.15.26:0/1369810539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0508143c70 con 0x7f05080af590 2024-04-26T21:10:23.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.283+0000 7f050d200640 1 -- 172.21.15.26:0/1369810539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0508144230 con 0x7f05080af590 2024-04-26T21:10:23.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.285+0000 7f050d200640 1 -- 172.21.15.26:0/1369810539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0508143a80 con 0x7f05080af590 2024-04-26T21:10:23.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.286+0000 7f04fe000640 1 -- 172.21.15.26:0/1369810539 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04f001a070 con 0x7f05080af590 2024-04-26T21:10:23.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.286+0000 7f04fe000640 1 --2- 172.21.15.26:0/1369810539 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f04e406c8f0 0x7f04e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:23.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.287+0000 7f04fe000640 1 -- 172.21.15.26:0/1369810539 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5338+0+0 (secure 0 0 0) 0x7f04f008bb60 con 0x7f05080af590 2024-04-26T21:10:23.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.287+0000 7f0505e00640 1 --2- 172.21.15.26:0/1369810539 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f04e406c8f0 0x7f04e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:23.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.287+0000 7f0505e00640 1 --2- 172.21.15.26:0/1369810539 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f04e406c8f0 0x7f04e406edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f04f8033000 tx=0x7f04f8007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:23.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.291+0000 7f04fe000640 1 -- 172.21.15.26:0/1369810539 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f04f005a060 con 0x7f05080af590 2024-04-26T21:10:23.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:23.496+0000 7f050d200640 1 -- 172.21.15.26:0/1369810539 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true} v 0) -- 0x7f050806a700 con 0x7f05080af590 2024-04-26T21:10:24.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.047+0000 7f04fe000640 1 -- 172.21.15.26:0/1369810539 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true}]=0 pool 'data_new' removed v41) ==== 160+0+0 (secure 0 0 0) 0x7f04f0059960 con 0x7f05080af590 2024-04-26T21:10:24.049 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'data_new' removed 2024-04-26T21:10:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.051+0000 7f050d200640 1 -- 172.21.15.26:0/1369810539 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f04e406c8f0 msgr2=0x7f04e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.051+0000 7f050d200640 1 --2- 172.21.15.26:0/1369810539 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f04e406c8f0 0x7f04e406edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f04f8033000 tx=0x7f04f8007000 comp rx=0 tx=0).stop 2024-04-26T21:10:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.051+0000 7f050d200640 1 -- 172.21.15.26:0/1369810539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05080af590 msgr2=0x7f050813d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.051+0000 7f050d200640 1 --2- 172.21.15.26:0/1369810539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05080af590 0x7f050813d630 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f04f000e7d0 tx=0x7f04f000ec90 comp rx=0 tx=0).stop 2024-04-26T21:10:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.052+0000 7f050d200640 1 -- 172.21.15.26:0/1369810539 shutdown_connections 2024-04-26T21:10:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.052+0000 7f050d200640 1 --2- 172.21.15.26:0/1369810539 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f04e406c8f0 0x7f04e406edb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.052+0000 7f050d200640 1 --2- 172.21.15.26:0/1369810539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05080b1170 0x7f05080b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.052+0000 7f050d200640 1 --2- 172.21.15.26:0/1369810539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05080af590 0x7f050813d630 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.052+0000 7f050d200640 1 --2- 172.21.15.26:0/1369810539 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04ec0039b0 0x7f05081431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.052+0000 7f050d200640 1 -- 172.21.15.26:0/1369810539 >> 172.21.15.26:0/1369810539 conn(0x7f050806d350 msgr2=0x7f0508147790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.052+0000 7f050d200640 1 -- 172.21.15.26:0/1369810539 shutdown_connections 2024-04-26T21:10:24.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.053+0000 7f050d200640 1 -- 172.21.15.26:0/1369810539 wait complete. 2024-04-26T21:10:24.080 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_symlink_recovery 2024-04-26T21:10:24.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.300+0000 7f0e5fe00640 1 Processor -- start 2024-04-26T21:10:24.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.300+0000 7f0e5fe00640 1 -- start start 2024-04-26T21:10:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.301+0000 7f0e5fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0e600b42e0 0x7f0e6006e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.301+0000 7f0e5fe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e600cfd40 0x7f0e6015ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.301+0000 7f0e5fe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e600b46c0 con 0x7f0e6006d560 2024-04-26T21:10:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.301+0000 7f0e5fe00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f0e600d2110 con 0x7f0e600b42e0 2024-04-26T21:10:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.301+0000 7f0e5fe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0e600d25a0 con 0x7f0e600cfd40 2024-04-26T21:10:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.302+0000 7f0e5ea00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0e600b42e0 0x7f0e6006e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.302+0000 7f0e5ea00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0e600b42e0 0x7f0e6006e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57180/0 (socket says 172.21.15.26:57180) 2024-04-26T21:10:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.302+0000 7f0e5ea00640 1 -- 172.21.15.26:0/854701786 learned_addr learned my addr 172.21.15.26:0/854701786 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.302+0000 7f0e5f400640 1 --2- 172.21.15.26:0/854701786 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e600cfd40 0x7f0e6015ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.303+0000 7f0e5ea00640 1 -- 172.21.15.26:0/854701786 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e600cfd40 msgr2=0x7f0e6015ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.303+0000 7f0e5ea00640 1 --2- 172.21.15.26:0/854701786 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e600cfd40 0x7f0e6015ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.303+0000 7f0e5ea00640 1 -- 172.21.15.26:0/854701786 >> v1:172.21.15.86:6789/0 conn(0x7f0e6006d560 legacy=0x7f0e600cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.303+0000 7f0e5ea00640 1 -- 172.21.15.26:0/854701786 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e600d1500 con 0x7f0e600b42e0 2024-04-26T21:10:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.304+0000 7f0e5ea00640 1 --2- 172.21.15.26:0/854701786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e600b42e0 0x7f0e6006e450 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009870 tx=0x7f0e5002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec08df9ac52c91f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.304+0000 7f0e5d600640 1 -- 172.21.15.26:0/854701786 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e50037070 con 0x7f0e600b42e0 2024-04-26T21:10:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.304+0000 7f0e5d600640 1 -- 172.21.15.26:0/854701786 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e50032440 con 0x7f0e600b42e0 2024-04-26T21:10:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.304+0000 7f0e5d600640 1 -- 172.21.15.26:0/854701786 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e50032810 con 0x7f0e600b42e0 2024-04-26T21:10:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.305+0000 7f0e5fe00640 1 -- 172.21.15.26:0/854701786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e600b42e0 msgr2=0x7f0e6006e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.305+0000 7f0e5fe00640 1 --2- 172.21.15.26:0/854701786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e600b42e0 0x7f0e6006e450 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009870 tx=0x7f0e5002a210 comp rx=0 tx=0).stop 2024-04-26T21:10:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.305+0000 7f0e5fe00640 1 -- 172.21.15.26:0/854701786 shutdown_connections 2024-04-26T21:10:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.305+0000 7f0e5fe00640 1 --2- 172.21.15.26:0/854701786 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e600cfd40 0x7f0e6015ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.305+0000 7f0e5fe00640 1 --2- 172.21.15.26:0/854701786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e600b42e0 0x7f0e6006e450 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.305+0000 7f0e5fe00640 1 -- 172.21.15.26:0/854701786 >> 172.21.15.26:0/854701786 conn(0x7f0e600d2db0 msgr2=0x7f0e600d31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.305+0000 7f0e5fe00640 1 -- 172.21.15.26:0/854701786 shutdown_connections 2024-04-26T21:10:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.305+0000 7f0e5fe00640 1 -- 172.21.15.26:0/854701786 wait complete. 2024-04-26T21:10:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.307+0000 7f0e5fe00640 1 Processor -- start 2024-04-26T21:10:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.307+0000 7f0e5fe00640 1 -- start start 2024-04-26T21:10:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.308+0000 7f0e5fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e6006d560 0x7f0e6015c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.308+0000 7f0e5fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e600b42e0 0x7f0e601f8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.308+0000 7f0e5fe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e600cfd40 0x7f0e60162500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.308+0000 7f0e5fe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0e600d1da0 con 0x7f0e600b42e0 2024-04-26T21:10:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.308+0000 7f0e5fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0e600d1ac0 con 0x7f0e6006d560 2024-04-26T21:10:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.308+0000 7f0e5fe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0e600d1120 con 0x7f0e600cfd40 2024-04-26T21:10:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.308+0000 7f0e5e000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e600b42e0 0x7f0e601f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.308+0000 7f0e5e000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e600b42e0 0x7f0e601f8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57196/0 (socket says 172.21.15.26:57196) 2024-04-26T21:10:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.308+0000 7f0e5e000640 1 -- 172.21.15.26:0/1889763482 learned_addr learned my addr 172.21.15.26:0/1889763482 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.308+0000 7f0e5e000640 1 -- 172.21.15.26:0/1889763482 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e600cfd40 msgr2=0x7f0e60162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.308+0000 7f0e5e000640 1 --2- 172.21.15.26:0/1889763482 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e600cfd40 0x7f0e60162500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.308+0000 7f0e5e000640 1 -- 172.21.15.26:0/1889763482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e6006d560 msgr2=0x7f0e6015c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.308+0000 7f0e5e000640 1 --2- 172.21.15.26:0/1889763482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e6006d560 0x7f0e6015c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.308+0000 7f0e5e000640 1 -- 172.21.15.26:0/1889763482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e601f8ef0 con 0x7f0e600b42e0 2024-04-26T21:10:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.309+0000 7f0e5e000640 1 --2- 172.21.15.26:0/1889763482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e600b42e0 0x7f0e601f8860 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00c430 tx=0x7f0e4c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:24.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.309+0000 7f0e56a00640 1 -- 172.21.15.26:0/1889763482 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c011790 con 0x7f0e600b42e0 2024-04-26T21:10:24.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.309+0000 7f0e56a00640 1 -- 172.21.15.26:0/1889763482 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e4c017070 con 0x7f0e600b42e0 2024-04-26T21:10:24.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.309+0000 7f0e5fe00640 1 -- 172.21.15.26:0/1889763482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e601f91d0 con 0x7f0e600b42e0 2024-04-26T21:10:24.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.309+0000 7f0e56a00640 1 -- 172.21.15.26:0/1889763482 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c011bb0 con 0x7f0e600b42e0 2024-04-26T21:10:24.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.309+0000 7f0e5fe00640 1 -- 172.21.15.26:0/1889763482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e6015ffb0 con 0x7f0e600b42e0 2024-04-26T21:10:24.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.311+0000 7f0e5fe00640 1 -- 172.21.15.26:0/1889763482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e601601b0 con 0x7f0e600b42e0 2024-04-26T21:10:24.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.311+0000 7f0e56a00640 1 -- 172.21.15.26:0/1889763482 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e4c01c460 con 0x7f0e600b42e0 2024-04-26T21:10:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.312+0000 7f0e56a00640 1 --2- 172.21.15.26:0/1889763482 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0e4006c8f0 0x7f0e4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.312+0000 7f0e56a00640 1 -- 172.21.15.26:0/1889763482 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f0e4c08b690 con 0x7f0e600b42e0 2024-04-26T21:10:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.313+0000 7f0e5ea00640 1 --2- 172.21.15.26:0/1889763482 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0e4006c8f0 0x7f0e4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.313+0000 7f0e5ea00640 1 --2- 172.21.15.26:0/1889763482 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0e4006c8f0 0x7f0e4006edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0e50034000 tx=0x7f0e5002c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:24.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.317+0000 7f0e56a00640 1 -- 172.21.15.26:0/1889763482 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0e601601b0 con 0x7f0e600b42e0 2024-04-26T21:10:24.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.521+0000 7f0e5fe00640 1 -- 172.21.15.26:0/1889763482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_symlink_recovery"} v 0) -- 0x7f0e600506c0 con 0x7f0e600b42e0 2024-04-26T21:10:24.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.529+0000 7f0e56a00640 1 -- 172.21.15.26:0/1889763482 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_symlink_recovery"}]=0 v13) ==== 103+0+0 (secure 0 0 0) 0x7f0e4c059e50 con 0x7f0e600b42e0 2024-04-26T21:10:24.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.532+0000 7f0e5fe00640 1 -- 172.21.15.26:0/1889763482 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0e4006c8f0 msgr2=0x7f0e4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:24.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.532+0000 7f0e5fe00640 1 --2- 172.21.15.26:0/1889763482 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0e4006c8f0 0x7f0e4006edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0e50034000 tx=0x7f0e5002c000 comp rx=0 tx=0).stop 2024-04-26T21:10:24.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.532+0000 7f0e5fe00640 1 -- 172.21.15.26:0/1889763482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e600b42e0 msgr2=0x7f0e601f8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:24.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.532+0000 7f0e5fe00640 1 --2- 172.21.15.26:0/1889763482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e600b42e0 0x7f0e601f8860 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00c430 tx=0x7f0e4c00c8f0 comp rx=0 tx=0).stop 2024-04-26T21:10:24.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.532+0000 7f0e5fe00640 1 -- 172.21.15.26:0/1889763482 shutdown_connections 2024-04-26T21:10:24.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.532+0000 7f0e5fe00640 1 --2- 172.21.15.26:0/1889763482 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0e4006c8f0 0x7f0e4006edb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:24.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.532+0000 7f0e5fe00640 1 --2- 172.21.15.26:0/1889763482 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e600cfd40 0x7f0e60162500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:24.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.532+0000 7f0e5fe00640 1 --2- 172.21.15.26:0/1889763482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e600b42e0 0x7f0e601f8860 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:24.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.532+0000 7f0e5fe00640 1 --2- 172.21.15.26:0/1889763482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e6006d560 0x7f0e6015c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:24.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.532+0000 7f0e5fe00640 1 -- 172.21.15.26:0/1889763482 >> 172.21.15.26:0/1889763482 conn(0x7f0e600d2db0 msgr2=0x7f0e600c7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:24.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.532+0000 7f0e5fe00640 1 -- 172.21.15.26:0/1889763482 shutdown_connections 2024-04-26T21:10:24.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.532+0000 7f0e5fe00640 1 -- 172.21.15.26:0/1889763482 wait complete. 2024-04-26T21:10:24.554 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T21:10:24.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.763+0000 7ff5bbe00640 1 Processor -- start 2024-04-26T21:10:24.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.764+0000 7ff5bbe00640 1 -- start start 2024-04-26T21:10:24.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.765+0000 7ff5bbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff5bc15f460 0x7ff5bc0d1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:24.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.765+0000 7ff5bbe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5bc0d2f30 0x7ff5bc0d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:24.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.765+0000 7ff5bbe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff5bc0d5b70 0x7ff5bc0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:24.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.765+0000 7ff5bbe00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7ff5bc0b49b0 con 0x7ff5bc0d5b70 2024-04-26T21:10:24.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.765+0000 7ff5bbe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff5bc0d1070 con 0x7ff5bc15f460 2024-04-26T21:10:24.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.765+0000 7ff5bbe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff5bc0b4c00 con 0x7ff5bc0d2f30 2024-04-26T21:10:24.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.765+0000 7ff5bb400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff5bc0d5b70 0x7ff5bc0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:24.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.765+0000 7ff5bb400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff5bc0d5b70 0x7ff5bc0cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57210/0 (socket says 172.21.15.26:57210) 2024-04-26T21:10:24.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.765+0000 7ff5bb400640 1 -- 172.21.15.26:0/3402300857 learned_addr learned my addr 172.21.15.26:0/3402300857 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:24.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.766+0000 7ff5ba000640 1 --2- 172.21.15.26:0/3402300857 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5bc0d2f30 0x7ff5bc0d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:24.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.766+0000 7ff5bb400640 1 -- 172.21.15.26:0/3402300857 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5bc0d2f30 msgr2=0x7ff5bc0d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:24.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.766+0000 7ff5baa00640 1 --2- 172.21.15.26:0/3402300857 >> v2:172.21.15.86:3300/0 conn(0x7ff5bc15f460 0x7ff5bc0d1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:24.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.766+0000 7ff5bb400640 1 --2- 172.21.15.26:0/3402300857 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5bc0d2f30 0x7ff5bc0d1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:24.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.766+0000 7ff5bb400640 1 -- 172.21.15.26:0/3402300857 >> v2:172.21.15.86:3300/0 conn(0x7ff5bc15f460 msgr2=0x7ff5bc0d1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:24.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.766+0000 7ff5bb400640 1 --2- 172.21.15.26:0/3402300857 >> v2:172.21.15.86:3300/0 conn(0x7ff5bc15f460 0x7ff5bc0d1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:24.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.766+0000 7ff5bb400640 1 -- 172.21.15.26:0/3402300857 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc0b4ee0 con 0x7ff5bc0d5b70 2024-04-26T21:10:24.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.767+0000 7ff5bb400640 1 --2- 172.21.15.26:0/3402300857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5bc0d5b70 0x7ff5bc0cfbf0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc08d3d0 tx=0x7ff5b402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c60bd1914ce819 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:24.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.767+0000 7ff5b9600640 1 -- 172.21.15.26:0/3402300857 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4036070 con 0x7ff5bc0d5b70 2024-04-26T21:10:24.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.767+0000 7ff5b9600640 1 -- 172.21.15.26:0/3402300857 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b4031600 con 0x7ff5bc0d5b70 2024-04-26T21:10:24.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.767+0000 7ff5b9600640 1 -- 172.21.15.26:0/3402300857 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40319d0 con 0x7ff5bc0d5b70 2024-04-26T21:10:24.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.768+0000 7ff5bbe00640 1 -- 172.21.15.26:0/3402300857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5bc0d5b70 msgr2=0x7ff5bc0cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:24.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.768+0000 7ff5bbe00640 1 --2- 172.21.15.26:0/3402300857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5bc0d5b70 0x7ff5bc0cfbf0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc08d3d0 tx=0x7ff5b402a610 comp rx=0 tx=0).stop 2024-04-26T21:10:24.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.768+0000 7ff5bbe00640 1 -- 172.21.15.26:0/3402300857 shutdown_connections 2024-04-26T21:10:24.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.768+0000 7ff5bbe00640 1 --2- 172.21.15.26:0/3402300857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5bc0d5b70 0x7ff5bc0cfbf0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:24.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.768+0000 7ff5bbe00640 1 --2- 172.21.15.26:0/3402300857 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5bc0d2f30 0x7ff5bc0d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:24.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.768+0000 7ff5bbe00640 1 --2- 172.21.15.26:0/3402300857 >> v2:172.21.15.86:3300/0 conn(0x7ff5bc15f460 0x7ff5bc0d1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:24.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.768+0000 7ff5bbe00640 1 -- 172.21.15.26:0/3402300857 >> 172.21.15.26:0/3402300857 conn(0x7ff5bc0d5050 msgr2=0x7ff5bc0d5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:24.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.768+0000 7ff5bbe00640 1 -- 172.21.15.26:0/3402300857 shutdown_connections 2024-04-26T21:10:24.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.768+0000 7ff5bbe00640 1 -- 172.21.15.26:0/3402300857 wait complete. 2024-04-26T21:10:24.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.770+0000 7ff5bbe00640 1 Processor -- start 2024-04-26T21:10:24.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.770+0000 7ff5bbe00640 1 -- start start 2024-04-26T21:10:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.771+0000 7ff5bbe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5bc0d2f30 0x7ff5bc1545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.771+0000 7ff5bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5bc0d5b70 0x7ff5bc162000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.771+0000 7ff5bbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5bc15f460 0x7ff5bc0b98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.771+0000 7ff5bbe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc15f9c0 con 0x7ff5bc15f460 2024-04-26T21:10:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.771+0000 7ff5bbe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc15f840 con 0x7ff5bc0d5b70 2024-04-26T21:10:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.771+0000 7ff5bbe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7ff5bc0d0130 con 0x7ff5bc0d2f30 2024-04-26T21:10:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.772+0000 7ff5bb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5bc15f460 0x7ff5bc0b98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.772+0000 7ff5bb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5bc15f460 0x7ff5bc0b98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57216/0 (socket says 172.21.15.26:57216) 2024-04-26T21:10:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.772+0000 7ff5bb400640 1 -- 172.21.15.26:0/4131421048 learned_addr learned my addr 172.21.15.26:0/4131421048 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.772+0000 7ff5ba000640 1 --2- 172.21.15.26:0/4131421048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5bc0d5b70 0x7ff5bc162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:24.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.772+0000 7ff5baa00640 1 --2- 172.21.15.26:0/4131421048 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5bc0d2f30 0x7ff5bc1545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:24.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.772+0000 7ff5bb400640 1 -- 172.21.15.26:0/4131421048 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5bc0d2f30 msgr2=0x7ff5bc1545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:24.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.772+0000 7ff5bb400640 1 --2- 172.21.15.26:0/4131421048 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5bc0d2f30 0x7ff5bc1545e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:24.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.772+0000 7ff5bb400640 1 -- 172.21.15.26:0/4131421048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5bc0d5b70 msgr2=0x7ff5bc162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:24.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.772+0000 7ff5bb400640 1 --2- 172.21.15.26:0/4131421048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5bc0d5b70 0x7ff5bc162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:24.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.772+0000 7ff5bb400640 1 -- 172.21.15.26:0/4131421048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc154b20 con 0x7ff5bc15f460 2024-04-26T21:10:24.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.772+0000 7ff5bb400640 1 --2- 172.21.15.26:0/4131421048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5bc15f460 0x7ff5bc0b98c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4034040 tx=0x7ff5b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:24.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.773+0000 7ff5b3400640 1 -- 172.21.15.26:0/4131421048 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4041020 con 0x7ff5bc15f460 2024-04-26T21:10:24.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.773+0000 7ff5b3400640 1 -- 172.21.15.26:0/4131421048 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b4007780 con 0x7ff5bc15f460 2024-04-26T21:10:24.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.773+0000 7ff5b3400640 1 -- 172.21.15.26:0/4131421048 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4036040 con 0x7ff5bc15f460 2024-04-26T21:10:24.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.773+0000 7ff5bbe00640 1 -- 172.21.15.26:0/4131421048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5bc0b8740 con 0x7ff5bc15f460 2024-04-26T21:10:24.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.773+0000 7ff5bbe00640 1 -- 172.21.15.26:0/4131421048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5bc0b89e0 con 0x7ff5bc15f460 2024-04-26T21:10:24.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.774+0000 7ff5bbe00640 1 -- 172.21.15.26:0/4131421048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5bc0506c0 con 0x7ff5bc15f460 2024-04-26T21:10:24.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.775+0000 7ff5b3400640 1 -- 172.21.15.26:0/4131421048 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5b4044020 con 0x7ff5bc15f460 2024-04-26T21:10:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.776+0000 7ff5b3400640 1 --2- 172.21.15.26:0/4131421048 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff59c06c8f0 0x7ff59c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:24.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.776+0000 7ff5b3400640 1 -- 172.21.15.26:0/4131421048 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7ff5b40ac700 con 0x7ff5bc15f460 2024-04-26T21:10:24.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.776+0000 7ff5baa00640 1 --2- 172.21.15.26:0/4131421048 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff59c06c8f0 0x7ff59c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:24.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.777+0000 7ff5baa00640 1 --2- 172.21.15.26:0/4131421048 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff59c06c8f0 0x7ff59c06edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac006fd0 tx=0x7ff5ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:24.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.781+0000 7ff5b3400640 1 -- 172.21.15.26:0/4131421048 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff5b404b030 con 0x7ff5bc15f460 2024-04-26T21:10:24.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.989+0000 7ff5bbe00640 1 -- 172.21.15.26:0/4131421048 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7ff5bc06de60 con 0x7ff5bc15f460 2024-04-26T21:10:24.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.994+0000 7ff5b3400640 1 -- 172.21.15.26:0/4131421048 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v14) ==== 104+0+0 (secure 0 0 0) 0x7ff5bc154b20 con 0x7ff5bc15f460 2024-04-26T21:10:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.997+0000 7ff5bbe00640 1 -- 172.21.15.26:0/4131421048 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff59c06c8f0 msgr2=0x7ff59c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.997+0000 7ff5bbe00640 1 --2- 172.21.15.26:0/4131421048 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff59c06c8f0 0x7ff59c06edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac006fd0 tx=0x7ff5ac008040 comp rx=0 tx=0).stop 2024-04-26T21:10:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.997+0000 7ff5bbe00640 1 -- 172.21.15.26:0/4131421048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5bc15f460 msgr2=0x7ff5bc0b98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.997+0000 7ff5bbe00640 1 --2- 172.21.15.26:0/4131421048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5bc15f460 0x7ff5bc0b98c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4034040 tx=0x7ff5b4004a40 comp rx=0 tx=0).stop 2024-04-26T21:10:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.997+0000 7ff5bbe00640 1 -- 172.21.15.26:0/4131421048 shutdown_connections 2024-04-26T21:10:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.997+0000 7ff5bbe00640 1 --2- 172.21.15.26:0/4131421048 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7ff59c06c8f0 0x7ff59c06edb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.997+0000 7ff5bbe00640 1 --2- 172.21.15.26:0/4131421048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5bc15f460 0x7ff5bc0b98c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.997+0000 7ff5bbe00640 1 --2- 172.21.15.26:0/4131421048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5bc0d5b70 0x7ff5bc162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.997+0000 7ff5bbe00640 1 --2- 172.21.15.26:0/4131421048 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5bc0d2f30 0x7ff5bc1545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.998+0000 7ff5bbe00640 1 -- 172.21.15.26:0/4131421048 >> 172.21.15.26:0/4131421048 conn(0x7ff5bc0d5050 msgr2=0x7ff5bc0d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.998+0000 7ff5bbe00640 1 -- 172.21.15.26:0/4131421048 shutdown_connections 2024-04-26T21:10:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:24.998+0000 7ff5bbe00640 1 -- 172.21.15.26:0/4131421048 wait complete. 2024-04-26T21:10:25.019 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace' 2024-04-26T21:10:25.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.219+0000 7f88a2800640 1 Processor -- start 2024-04-26T21:10:25.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.220+0000 7f88a2800640 1 -- start start 2024-04-26T21:10:25.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.221+0000 7f88a2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f889c15f340 0x7f889c0d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:25.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.221+0000 7f88a2800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f889c0d3d70 0x7f889c0b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:25.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.221+0000 7f88a2800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f889c0d01d0 con 0x7f889c0d2270 2024-04-26T21:10:25.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.221+0000 7f88a2800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f889c0d04b0 con 0x7f889c15f340 2024-04-26T21:10:25.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.221+0000 7f88a2800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f889c0d0790 con 0x7f889c0d3d70 2024-04-26T21:10:25.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.222+0000 7f88a1400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f889c15f340 0x7f889c0d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.222+0000 7f88a1400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f889c15f340 0x7f889c0d1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.26:59446/0 (socket says 172.21.15.26:59446) 2024-04-26T21:10:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.222+0000 7f88a1400640 1 -- 172.21.15.26:0/1773373724 learned_addr learned my addr 172.21.15.26:0/1773373724 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.222+0000 7f88a1e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f889c0d3d70 0x7f889c0b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.223+0000 7f88a1400640 1 -- 172.21.15.26:0/1773373724 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f889c0d3d70 msgr2=0x7f889c0b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.223+0000 7f88a1400640 1 --2- 172.21.15.26:0/1773373724 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f889c0d3d70 0x7f889c0b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.223+0000 7f88a1400640 1 -- 172.21.15.26:0/1773373724 >> v1:172.21.15.86:6789/0 conn(0x7f889c0d2270 legacy=0x7f889c0d3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.223+0000 7f88a1400640 1 -- 172.21.15.26:0/1773373724 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889c0d0a70 con 0x7f889c15f340 2024-04-26T21:10:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.223+0000 7f889be00640 1 -- 172.21.15.26:0/1773373724 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 627544667 0 0) 0x7f889c0d01d0 con 0x7f889c0d2270 2024-04-26T21:10:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.224+0000 7f88a1400640 1 --2- 172.21.15.26:0/1773373724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f889c15f340 0x7f889c0d1d30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8890009870 tx=0x7f889002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caaf051726359e0a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.225+0000 7f889be00640 1 -- 172.21.15.26:0/1773373724 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890037070 con 0x7f889c15f340 2024-04-26T21:10:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.225+0000 7f889be00640 1 -- 172.21.15.26:0/1773373724 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8890032440 con 0x7f889c15f340 2024-04-26T21:10:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.225+0000 7f889be00640 1 -- 172.21.15.26:0/1773373724 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890032810 con 0x7f889c15f340 2024-04-26T21:10:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.225+0000 7f88a2800640 1 -- 172.21.15.26:0/1773373724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f889c15f340 msgr2=0x7f889c0d1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.225+0000 7f88a2800640 1 --2- 172.21.15.26:0/1773373724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f889c15f340 0x7f889c0d1d30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8890009870 tx=0x7f889002a250 comp rx=0 tx=0).stop 2024-04-26T21:10:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.226+0000 7f88a2800640 1 -- 172.21.15.26:0/1773373724 shutdown_connections 2024-04-26T21:10:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.226+0000 7f88a2800640 1 --2- 172.21.15.26:0/1773373724 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f889c0d3d70 0x7f889c0b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.226+0000 7f88a2800640 1 --2- 172.21.15.26:0/1773373724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f889c15f340 0x7f889c0d1d30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.226+0000 7f88a2800640 1 -- 172.21.15.26:0/1773373724 >> 172.21.15.26:0/1773373724 conn(0x7f889c09e370 msgr2=0x7f889c0d50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:25.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.226+0000 7f88a2800640 1 -- 172.21.15.26:0/1773373724 shutdown_connections 2024-04-26T21:10:25.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.226+0000 7f88a2800640 1 -- 172.21.15.26:0/1773373724 wait complete. 2024-04-26T21:10:25.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.228+0000 7f88a2800640 1 Processor -- start 2024-04-26T21:10:25.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.228+0000 7f88a2800640 1 -- start start 2024-04-26T21:10:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.230+0000 7f88a2800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c0d2270 0x7f889c154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.230+0000 7f88a2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f889c0d3d70 0x7f889c1f2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.230+0000 7f88a2800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f889c15f340 0x7f889c1f8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.230+0000 7f88a2800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f889c0cfc40 con 0x7f889c0d2270 2024-04-26T21:10:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.230+0000 7f88a2800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f889c15f720 con 0x7f889c0d3d70 2024-04-26T21:10:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.230+0000 7f88a2800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f889c0cfdc0 con 0x7f889c15f340 2024-04-26T21:10:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.230+0000 7f88a1400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c0d2270 0x7f889c154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.230+0000 7f88a1400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c0d2270 0x7f889c154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57228/0 (socket says 172.21.15.26:57228) 2024-04-26T21:10:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.230+0000 7f88a1400640 1 -- 172.21.15.26:0/1206013936 learned_addr learned my addr 172.21.15.26:0/1206013936 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.230+0000 7f88a0a00640 1 --2- 172.21.15.26:0/1206013936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f889c0d3d70 0x7f889c1f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.230+0000 7f88a1e00640 1 --2- 172.21.15.26:0/1206013936 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f889c15f340 0x7f889c1f8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.231+0000 7f88a1400640 1 -- 172.21.15.26:0/1206013936 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f889c15f340 msgr2=0x7f889c1f8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.231+0000 7f88a1400640 1 --2- 172.21.15.26:0/1206013936 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f889c15f340 0x7f889c1f8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.231+0000 7f88a1400640 1 -- 172.21.15.26:0/1206013936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f889c0d3d70 msgr2=0x7f889c1f2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.231+0000 7f88a1400640 1 --2- 172.21.15.26:0/1206013936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f889c0d3d70 0x7f889c1f2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.231+0000 7f88a1400640 1 -- 172.21.15.26:0/1206013936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889c156050 con 0x7f889c0d2270 2024-04-26T21:10:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.231+0000 7f88a1400640 1 --2- 172.21.15.26:0/1206013936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c0d2270 0x7f889c154290 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8890000c00 tx=0x7f88900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.231+0000 7f8899600640 1 -- 172.21.15.26:0/1206013936 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890037040 con 0x7f889c0d2270 2024-04-26T21:10:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.231+0000 7f8899600640 1 -- 172.21.15.26:0/1206013936 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8890042070 con 0x7f889c0d2270 2024-04-26T21:10:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.231+0000 7f8899600640 1 -- 172.21.15.26:0/1206013936 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890030880 con 0x7f889c0d2270 2024-04-26T21:10:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.231+0000 7f88a2800640 1 -- 172.21.15.26:0/1206013936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f889c156330 con 0x7f889c0d2270 2024-04-26T21:10:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.232+0000 7f88a2800640 1 -- 172.21.15.26:0/1206013936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f889c1568f0 con 0x7f889c0d2270 2024-04-26T21:10:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.233+0000 7f88a2800640 1 -- 172.21.15.26:0/1206013936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f889c06a700 con 0x7f889c0d2270 2024-04-26T21:10:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.234+0000 7f8899600640 1 -- 172.21.15.26:0/1206013936 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8890030a20 con 0x7f889c0d2270 2024-04-26T21:10:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.235+0000 7f8899600640 1 --2- 172.21.15.26:0/1206013936 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f887c06c8f0 0x7f887c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.235+0000 7f8899600640 1 -- 172.21.15.26:0/1206013936 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f88900ac0a0 con 0x7f889c0d2270 2024-04-26T21:10:25.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.235+0000 7f88a0a00640 1 --2- 172.21.15.26:0/1206013936 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f887c06c8f0 0x7f887c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:25.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.236+0000 7f88a0a00640 1 --2- 172.21.15.26:0/1206013936 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f887c06c8f0 0x7f887c06edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f889c1628f0 tx=0x7f8888009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.241+0000 7f8899600640 1 -- 172.21.15.26:0/1206013936 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f889007a7c0 con 0x7f889c0d2270 2024-04-26T21:10:25.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:25.520+0000 7f88a2800640 1 -- 172.21.15.26:0/1206013936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace"]} v 0) -- 0x7f889c0d26b0 con 0x7f889c0d2270 2024-04-26T21:10:26.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:26.061+0000 7f8899600640 1 -- 172.21.15.26:0/1206013936 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace"]}]=0 v120) ==== 135+0+0 (secure 0 0 0) 0x7f889007a0c0 con 0x7f889c0d2270 2024-04-26T21:10:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:26.064+0000 7f88a2800640 1 -- 172.21.15.26:0/1206013936 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f887c06c8f0 msgr2=0x7f887c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:26.064+0000 7f88a2800640 1 --2- 172.21.15.26:0/1206013936 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f887c06c8f0 0x7f887c06edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f889c1628f0 tx=0x7f8888009040 comp rx=0 tx=0).stop 2024-04-26T21:10:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:26.064+0000 7f88a2800640 1 -- 172.21.15.26:0/1206013936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c0d2270 msgr2=0x7f889c154290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:26.065+0000 7f88a2800640 1 --2- 172.21.15.26:0/1206013936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c0d2270 0x7f889c154290 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8890000c00 tx=0x7f88900046c0 comp rx=0 tx=0).stop 2024-04-26T21:10:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:26.065+0000 7f88a2800640 1 -- 172.21.15.26:0/1206013936 shutdown_connections 2024-04-26T21:10:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:26.065+0000 7f88a2800640 1 --2- 172.21.15.26:0/1206013936 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f887c06c8f0 0x7f887c06edb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:26.065+0000 7f88a2800640 1 --2- 172.21.15.26:0/1206013936 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f889c15f340 0x7f889c1f8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:26.065+0000 7f88a2800640 1 --2- 172.21.15.26:0/1206013936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f889c0d3d70 0x7f889c1f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:26.065+0000 7f88a2800640 1 --2- 172.21.15.26:0/1206013936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c0d2270 0x7f889c154290 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:26.065+0000 7f88a2800640 1 -- 172.21.15.26:0/1206013936 >> 172.21.15.26:0/1206013936 conn(0x7f889c09e370 msgr2=0x7f889c15cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:26.065+0000 7f88a2800640 1 -- 172.21.15.26:0/1206013936 shutdown_connections 2024-04-26T21:10:26.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:26.065+0000 7f88a2800640 1 -- 172.21.15.26:0/1206013936 wait complete. 2024-04-26T21:10:26.087 INFO:tasks.cephfs_test_runner:That the 'parent' 'layout' and 'symlink' xattrs on the head objects of files ... ok 2024-04-26T21:10:26.087 INFO:tasks.cephfs_test_runner: 2024-04-26T21:10:26.087 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-26T21:10:26.088 INFO:tasks.cephfs_test_runner:Ran 1 test in 67.969s 2024-04-26T21:10:26.088 INFO:tasks.cephfs_test_runner: 2024-04-26T21:10:26.088 INFO:tasks.cephfs_test_runner:OK 2024-04-26T21:10:26.088 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-26T21:10:26.098 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-26T21:10:26.107 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-26T21:10:26.107 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:10:26.107 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:10:26.165 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:10:26.165 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-04-26T21:10:26.165 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:10:26.221 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:10:26.221 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:10:26.221 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:10:26.279 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:10:26.279 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-26T21:10:26.279 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:10:26.333 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:10:26.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:10:26.334 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-04-26T21:10:26.395 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:10:26.395 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:10:26.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:10:26.397 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:10:26.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:10:26.489 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:10:26.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:10:26.567 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-04-26T21:10:26.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:10:26.712 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-04-26T21:10:26.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:10:26.754 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-04-26T21:10:26.883 INFO:teuthology.orchestra.run.smithi026.stderr:Cannot find device "ceph-brx" 2024-04-26T21:10:26.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:10:26.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:10:26.887 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-26T21:10:26.944 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T21:10:26.944 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T21:10:26.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:10:26.945 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:10:27.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:10:27.035 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:10:27.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:10:27.106 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-26T21:10:27.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:10:27.234 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-26T21:10:27.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:10:27.277 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-26T21:10:27.400 INFO:teuthology.orchestra.run.smithi086.stderr:Cannot find device "ceph-brx" 2024-04-26T21:10:27.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:10:27.403 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-26T21:10:27.412 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T21:10:27.412 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:10:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.630+0000 7f55e9000640 1 Processor -- start 2024-04-26T21:10:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.631+0000 7f55e9000640 1 -- start start 2024-04-26T21:10:27.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.632+0000 7f55e9000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55e406ec20 0x7f55e40d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:27.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.632+0000 7f55e9000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f55e40d1210 0x7f55e4072100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:27.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.632+0000 7f55e9000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55e4073b70 con 0x7f55e4146990 2024-04-26T21:10:27.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.632+0000 7f55e9000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f55e40d1660 con 0x7f55e40d1210 2024-04-26T21:10:27.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.632+0000 7f55e9000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f55e4146d70 con 0x7f55e406ec20 2024-04-26T21:10:27.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.632+0000 7f55e1e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f55e4146990 0x7f55e40b3f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54618/0 (socket says 172.21.15.26:54618) 2024-04-26T21:10:27.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.632+0000 7f55e1e00640 1 -- 172.21.15.26:0/12704633 learned_addr learned my addr 172.21.15.26:0/12704633 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:27.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.632+0000 7f55e2800640 1 --2- 172.21.15.26:0/12704633 >> v2:172.21.15.26:3300/0 conn(0x7f55e40d1210 0x7f55e4072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:27.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.633+0000 7f55e0a00640 1 -- 172.21.15.26:0/12704633 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 444079887 0 0) 0x7f55e4073b70 con 0x7f55e4146990 2024-04-26T21:10:27.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.633+0000 7f55e0a00640 1 -- 172.21.15.26:0/12704633 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55c8003520 con 0x7f55e4146990 2024-04-26T21:10:27.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.633+0000 7f55e1400640 1 --2- 172.21.15.26:0/12704633 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55e406ec20 0x7f55e40d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:27.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.633+0000 7f55e2800640 1 -- 172.21.15.26:0/12704633 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55e406ec20 msgr2=0x7f55e40d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:27.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.633+0000 7f55e2800640 1 --2- 172.21.15.26:0/12704633 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55e406ec20 0x7f55e40d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:27.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.633+0000 7f55e2800640 1 -- 172.21.15.26:0/12704633 >> v1:172.21.15.26:6789/0 conn(0x7f55e4146990 legacy=0x7f55e40b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:27.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.633+0000 7f55e0a00640 1 -- 172.21.15.26:0/12704633 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2411674379 0 0) 0x7f55c8003520 con 0x7f55e4146990 2024-04-26T21:10:27.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.633+0000 7f55e2800640 1 -- 172.21.15.26:0/12704633 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e4073e50 con 0x7f55e40d1210 2024-04-26T21:10:27.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.634+0000 7f55e2800640 1 --2- 172.21.15.26:0/12704633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e40d1210 0x7f55e4072100 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0097a0 tx=0x7f55dc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea9bd5ad8f70078 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:27.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.634+0000 7f55e0a00640 1 -- 172.21.15.26:0/12704633 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc037070 con 0x7f55e40d1210 2024-04-26T21:10:27.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.634+0000 7f55e0a00640 1 -- 172.21.15.26:0/12704633 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55dc02aba0 con 0x7f55e40d1210 2024-04-26T21:10:27.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.634+0000 7f55e0a00640 1 -- 172.21.15.26:0/12704633 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc001670 con 0x7f55e40d1210 2024-04-26T21:10:27.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.635+0000 7f55e9000640 1 -- 172.21.15.26:0/12704633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e40d1210 msgr2=0x7f55e4072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:27.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.635+0000 7f55e9000640 1 --2- 172.21.15.26:0/12704633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e40d1210 0x7f55e4072100 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0097a0 tx=0x7f55dc02a110 comp rx=0 tx=0).stop 2024-04-26T21:10:27.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.635+0000 7f55e9000640 1 -- 172.21.15.26:0/12704633 shutdown_connections 2024-04-26T21:10:27.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.635+0000 7f55e9000640 1 --2- 172.21.15.26:0/12704633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e40d1210 0x7f55e4072100 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:27.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.635+0000 7f55e9000640 1 --2- 172.21.15.26:0/12704633 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55e406ec20 0x7f55e40d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:27.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.635+0000 7f55e9000640 1 -- 172.21.15.26:0/12704633 >> 172.21.15.26:0/12704633 conn(0x7f55e40cd840 msgr2=0x7f55e40cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:27.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.635+0000 7f55e9000640 1 -- 172.21.15.26:0/12704633 shutdown_connections 2024-04-26T21:10:27.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.635+0000 7f55e9000640 1 -- 172.21.15.26:0/12704633 wait complete. 2024-04-26T21:10:27.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.637+0000 7f55e9000640 1 Processor -- start 2024-04-26T21:10:27.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.637+0000 7f55e9000640 1 -- start start 2024-04-26T21:10:27.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.638+0000 7f55e9000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55e406ec20 0x7f55e4153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:27.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.638+0000 7f55e9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e40d1210 0x7f55e40b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:27.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.638+0000 7f55e9000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e4146990 0x7f55e40b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:27.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.638+0000 7f55e9000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f55e406e700 con 0x7f55e4146990 2024-04-26T21:10:27.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.638+0000 7f55e9000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55e406e420 con 0x7f55e40d1210 2024-04-26T21:10:27.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.638+0000 7f55e9000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f55e406f0b0 con 0x7f55e406ec20 2024-04-26T21:10:27.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.639+0000 7f55e2800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e4146990 0x7f55e40b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:27.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.639+0000 7f55e2800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e4146990 0x7f55e40b2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57248/0 (socket says 172.21.15.26:57248) 2024-04-26T21:10:27.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.639+0000 7f55e2800640 1 -- 172.21.15.26:0/2491534171 learned_addr learned my addr 172.21.15.26:0/2491534171 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:27.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.639+0000 7f55e1e00640 1 --2- 172.21.15.26:0/2491534171 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55e406ec20 0x7f55e4153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:27.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.639+0000 7f55e1400640 1 --2- 172.21.15.26:0/2491534171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e40d1210 0x7f55e40b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:27.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.639+0000 7f55e2800640 1 -- 172.21.15.26:0/2491534171 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55e406ec20 msgr2=0x7f55e4153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:27.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.639+0000 7f55e2800640 1 --2- 172.21.15.26:0/2491534171 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55e406ec20 0x7f55e4153370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:27.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.639+0000 7f55e2800640 1 -- 172.21.15.26:0/2491534171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e40d1210 msgr2=0x7f55e40b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:27.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.639+0000 7f55e2800640 1 --2- 172.21.15.26:0/2491534171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e40d1210 0x7f55e40b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:27.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.639+0000 7f55e2800640 1 -- 172.21.15.26:0/2491534171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e4143160 con 0x7f55e4146990 2024-04-26T21:10:27.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.639+0000 7f55e1400640 1 --2- 172.21.15.26:0/2491534171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e40d1210 0x7f55e40b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:27.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.639+0000 7f55e2800640 1 --2- 172.21.15.26:0/2491534171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e4146990 0x7f55e40b2120 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f55dc035040 tx=0x7f55dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:27.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.640+0000 7f55da000640 1 -- 172.21.15.26:0/2491534171 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc041020 con 0x7f55e4146990 2024-04-26T21:10:27.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.640+0000 7f55da000640 1 -- 172.21.15.26:0/2491534171 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55dc03d620 con 0x7f55e4146990 2024-04-26T21:10:27.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.640+0000 7f55da000640 1 -- 172.21.15.26:0/2491534171 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc037040 con 0x7f55e4146990 2024-04-26T21:10:27.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.640+0000 7f55e9000640 1 -- 172.21.15.26:0/2491534171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e4143440 con 0x7f55e4146990 2024-04-26T21:10:27.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.640+0000 7f55e9000640 1 -- 172.21.15.26:0/2491534171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e415f610 con 0x7f55e4146990 2024-04-26T21:10:27.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.641+0000 7f55e9000640 1 -- 172.21.15.26:0/2491534171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55e415f120 con 0x7f55e4146990 2024-04-26T21:10:27.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.642+0000 7f55da000640 1 -- 172.21.15.26:0/2491534171 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f55dc044020 con 0x7f55e4146990 2024-04-26T21:10:27.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.643+0000 7f55da000640 1 --2- 172.21.15.26:0/2491534171 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f55c006c8f0 0x7f55c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:27.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.643+0000 7f55da000640 1 -- 172.21.15.26:0/2491534171 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f55dc0abda0 con 0x7f55e4146990 2024-04-26T21:10:27.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.643+0000 7f55e1e00640 1 --2- 172.21.15.26:0/2491534171 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f55c006c8f0 0x7f55c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:27.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.644+0000 7f55e1e00640 1 --2- 172.21.15.26:0/2491534171 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f55c006c8f0 0x7f55c006edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f55cc005880 tx=0x7f55cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:27.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.648+0000 7f55da000640 1 -- 172.21.15.26:0/2491534171 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f55dc04b030 con 0x7f55e4146990 2024-04-26T21:10:27.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.847+0000 7f55e9000640 1 -- 172.21.15.26:0/2491534171 --> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f55e415f850 con 0x7f55c006c8f0 2024-04-26T21:10:27.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.849+0000 7f55da000640 1 -- 172.21.15.26:0/2491534171 <== mgr.4104 v2:172.21.15.26:6832/3130430358 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13495 (secure 0 0 0) 0x7f55cc001450 con 0x7f55c006c8f0 2024-04-26T21:10:27.850 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:27.851 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-26T21:10:27.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.851+0000 7f55e9000640 1 -- 172.21.15.26:0/2491534171 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f55c006c8f0 msgr2=0x7f55c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:27.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.851+0000 7f55e9000640 1 --2- 172.21.15.26:0/2491534171 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f55c006c8f0 0x7f55c006edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f55cc005880 tx=0x7f55cc009040 comp rx=0 tx=0).stop 2024-04-26T21:10:27.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.851+0000 7f55e9000640 1 -- 172.21.15.26:0/2491534171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e4146990 msgr2=0x7f55e40b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:27.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.851+0000 7f55e9000640 1 --2- 172.21.15.26:0/2491534171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e4146990 0x7f55e40b2120 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f55dc035040 tx=0x7f55dc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:10:27.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.851+0000 7f55e9000640 1 -- 172.21.15.26:0/2491534171 shutdown_connections 2024-04-26T21:10:27.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.851+0000 7f55e9000640 1 --2- 172.21.15.26:0/2491534171 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f55c006c8f0 0x7f55c006edb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:27.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.851+0000 7f55e9000640 1 --2- 172.21.15.26:0/2491534171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e4146990 0x7f55e40b2120 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:27.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.851+0000 7f55e9000640 1 --2- 172.21.15.26:0/2491534171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55e40d1210 0x7f55e40b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:27.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.851+0000 7f55e9000640 1 --2- 172.21.15.26:0/2491534171 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55e406ec20 0x7f55e4153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:27.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.851+0000 7f55e9000640 1 -- 172.21.15.26:0/2491534171 >> 172.21.15.26:0/2491534171 conn(0x7f55e40cd840 msgr2=0x7f55e40cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:27.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.852+0000 7f55e9000640 1 -- 172.21.15.26:0/2491534171 shutdown_connections 2024-04-26T21:10:27.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:27.852+0000 7f55e9000640 1 -- 172.21.15.26:0/2491534171 wait complete. 2024-04-26T21:10:27.873 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2024-04-26T21:10:27.325258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":224,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":237692,"kb_used_data":4856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737028,"statfs":{"total":773094113280,"available":772850716672,"internally_reserved":0,"allocated":4972544,"data_stored":2718830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-59914,"num_objects":-33,"num_object_clones":0,"num_object_copies":-66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-33,"num_whiteouts":0,"num_read":-22,"num_read_kb":-11,"num_write":-192,"num_write_kb":-141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001923"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":207,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T21:10:24.056599+0000","last_change":"2024-04-26T21:08:33.504911+0000","last_active":"2024-04-26T21:10:24.056599+0000","last_peered":"2024-04-26T21:10:24.056599+0000","last_clean":"2024-04-26T21:10:24.056599+0000","last_became_active":"2024-04-26T21:08:33.503667+0000","last_became_peered":"2024-04-26T21:08:33.503667+0000","last_unstale":"2024-04-26T21:10:24.056599+0000","last_undegraded":"2024-04-26T21:10:24.056599+0000","last_fullsized":"2024-04-26T21:10:24.056599+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:32.013907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:32.013907+0000","last_clean_scrub_stamp":"2024-04-26T21:08:32.013907+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:09.596153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1077248,"data_stored":786096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27496,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":483328,"data_stored":195728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":466944,"data_stored":190182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":190182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31636,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":466944,"data_stored":190182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31656,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340184,"statfs":{"total":96636764160,"available":96604348416,"internally_reserved":0,"allocated":483328,"data_stored":195728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":466944,"data_stored":190182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1060864,"data_stored":780550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:10:27.874 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:10:28.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.085+0000 7f56f7e00640 1 Processor -- start 2024-04-26T21:10:28.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.085+0000 7f56f7e00640 1 -- start start 2024-04-26T21:10:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.087+0000 7f56f7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f56f815dcb0 0x7f56f8164590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.087+0000 7f56f7e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56f806e700 0x7f56f806ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.087+0000 7f56f7e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56f8164ad0 con 0x7f56f81677a0 2024-04-26T21:10:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.087+0000 7f56f7e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f56f815e090 con 0x7f56f815dcb0 2024-04-26T21:10:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.087+0000 7f56f7e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f56f8072500 con 0x7f56f806e700 2024-04-26T21:10:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.087+0000 7f56f7400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f56f81677a0 0x7f56f80d1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54628/0 (socket says 172.21.15.26:54628) 2024-04-26T21:10:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.088+0000 7f56f7400640 1 -- 172.21.15.26:0/2852510777 learned_addr learned my addr 172.21.15.26:0/2852510777 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.088+0000 7f56f6a00640 1 --2- 172.21.15.26:0/2852510777 >> v2:172.21.15.86:3300/0 conn(0x7f56f815dcb0 0x7f56f8164590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.088+0000 7f56f6000640 1 --2- 172.21.15.26:0/2852510777 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56f806e700 0x7f56f806ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.088+0000 7f56f5600640 1 -- 172.21.15.26:0/2852510777 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3443398456 0 0) 0x7f56f8164ad0 con 0x7f56f81677a0 2024-04-26T21:10:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.088+0000 7f56f5600640 1 -- 172.21.15.26:0/2852510777 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56dc003520 con 0x7f56f81677a0 2024-04-26T21:10:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.088+0000 7f56f5600640 1 -- 172.21.15.26:0/2852510777 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3386045086 0 0) 0x7f56dc003520 con 0x7f56f81677a0 2024-04-26T21:10:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.088+0000 7f56f5600640 1 -- 172.21.15.26:0/2852510777 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56f806e700 msgr2=0x7f56f806ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.088+0000 7f56f5600640 1 --2- 172.21.15.26:0/2852510777 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56f806e700 0x7f56f806ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.088+0000 7f56f5600640 1 -- 172.21.15.26:0/2852510777 >> v2:172.21.15.86:3300/0 conn(0x7f56f815dcb0 msgr2=0x7f56f8164590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.088+0000 7f56f5600640 1 --2- 172.21.15.26:0/2852510777 >> v2:172.21.15.86:3300/0 conn(0x7f56f815dcb0 0x7f56f8164590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.089+0000 7f56f5600640 1 -- 172.21.15.26:0/2852510777 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f80727e0 con 0x7f56f81677a0 2024-04-26T21:10:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.089+0000 7f56f5600640 1 -- 172.21.15.26:0/2852510777 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f56f00019f0 con 0x7f56f81677a0 2024-04-26T21:10:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.089+0000 7f56f5600640 1 -- 172.21.15.26:0/2852510777 >> v1:172.21.15.26:6789/0 conn(0x7f56f81677a0 legacy=0x7f56f80d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.089+0000 7f56f5600640 1 --2- 172.21.15.26:0/2852510777 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56f806e700 0x7f56dc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.089+0000 7f56f5600640 1 --2- 172.21.15.26:0/2852510777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56dc003f30 0x7f56dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.089+0000 7f56f5600640 1 --2- 172.21.15.26:0/2852510777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56dc004d40 0x7f56dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.089+0000 7f56f5600640 1 -- 172.21.15.26:0/2852510777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f56f8072500 con 0x7f56dc004d40 2024-04-26T21:10:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.089+0000 7f56f5600640 1 -- 172.21.15.26:0/2852510777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc00ba20 con 0x7f56dc003f30 2024-04-26T21:10:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.089+0000 7f56f5600640 1 -- 172.21.15.26:0/2852510777 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f56dc00bc20 con 0x7f56f806e700 2024-04-26T21:10:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.089+0000 7f56f7400640 1 --2- 172.21.15.26:0/2852510777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56dc004d40 0x7f56dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.089+0000 7f56f6000640 1 --2- 172.21.15.26:0/2852510777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56dc003f30 0x7f56dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.089+0000 7f56f6a00640 1 --2- 172.21.15.26:0/2852510777 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56f806e700 0x7f56dc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.090+0000 7f56f7400640 1 -- 172.21.15.26:0/2852510777 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56f806e700 msgr2=0x7f56dc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.090+0000 7f56f7400640 1 --2- 172.21.15.26:0/2852510777 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56f806e700 0x7f56dc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.090+0000 7f56f7400640 1 -- 172.21.15.26:0/2852510777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56dc003f30 msgr2=0x7f56dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:28.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.090+0000 7f56f7400640 1 --2- 172.21.15.26:0/2852510777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56dc003f30 0x7f56dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.090+0000 7f56f7400640 1 -- 172.21.15.26:0/2852510777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56dc003520 con 0x7f56dc004d40 2024-04-26T21:10:28.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.091+0000 7f56f7400640 1 --2- 172.21.15.26:0/2852510777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56dc004d40 0x7f56dc0051c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f56f000a8a0 tx=0x7f56dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab673cc7dff4a6d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:28.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.091+0000 7f56f5600640 1 -- 172.21.15.26:0/2852510777 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f000a4c0 con 0x7f56dc004d40 2024-04-26T21:10:28.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.091+0000 7f56f5600640 1 -- 172.21.15.26:0/2852510777 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56f002f070 con 0x7f56dc004d40 2024-04-26T21:10:28.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.091+0000 7f56f5600640 1 -- 172.21.15.26:0/2852510777 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f002bd70 con 0x7f56dc004d40 2024-04-26T21:10:28.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.092+0000 7f56f7e00640 1 -- 172.21.15.26:0/2852510777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56dc004d40 msgr2=0x7f56dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:28.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.092+0000 7f56f7e00640 1 --2- 172.21.15.26:0/2852510777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56dc004d40 0x7f56dc0051c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f56f000a8a0 tx=0x7f56dc005830 comp rx=0 tx=0).stop 2024-04-26T21:10:28.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.092+0000 7f56f7400640 1 -- 172.21.15.26:0/2852510777 reap_dead start 2024-04-26T21:10:28.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.092+0000 7f56f7e00640 1 -- 172.21.15.26:0/2852510777 shutdown_connections 2024-04-26T21:10:28.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.092+0000 7f56f7e00640 1 -- 172.21.15.26:0/2852510777 >> 172.21.15.26:0/2852510777 conn(0x7f56f806c690 msgr2=0x7f56f806caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:28.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.092+0000 7f56f7e00640 1 -- 172.21.15.26:0/2852510777 shutdown_connections 2024-04-26T21:10:28.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.092+0000 7f56f7e00640 1 -- 172.21.15.26:0/2852510777 wait complete. 2024-04-26T21:10:28.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.094+0000 7f56f7e00640 1 Processor -- start 2024-04-26T21:10:28.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.094+0000 7f56f7e00640 1 -- start start 2024-04-26T21:10:28.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.095+0000 7f56f7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56f8145dc0 0x7f56f81fd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.095+0000 7f56f7e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56f81461a0 0x7f56f815bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.095+0000 7f56f7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56f8148ea0 0x7f56f815c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.095+0000 7f56f7e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f56f8072da0 con 0x7f56f8148ea0 2024-04-26T21:10:28.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.095+0000 7f56f7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56f8167280 con 0x7f56f8145dc0 2024-04-26T21:10:28.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.095+0000 7f56f7e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f56f806e3c0 con 0x7f56f81461a0 2024-04-26T21:10:28.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.095+0000 7f56f7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56f8148ea0 0x7f56f815c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.095+0000 7f56f7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56f8148ea0 0x7f56f815c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57264/0 (socket says 172.21.15.26:57264) 2024-04-26T21:10:28.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.095+0000 7f56f7400640 1 -- 172.21.15.26:0/828520121 learned_addr learned my addr 172.21.15.26:0/828520121 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:28.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.095+0000 7f56f6000640 1 --2- 172.21.15.26:0/828520121 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56f81461a0 0x7f56f815bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.095+0000 7f56f6a00640 1 --2- 172.21.15.26:0/828520121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56f8145dc0 0x7f56f81fd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.095+0000 7f56f7400640 1 -- 172.21.15.26:0/828520121 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56f81461a0 msgr2=0x7f56f815bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:28.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.095+0000 7f56f7400640 1 --2- 172.21.15.26:0/828520121 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56f81461a0 0x7f56f815bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.095+0000 7f56f7400640 1 -- 172.21.15.26:0/828520121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56f8145dc0 msgr2=0x7f56f81fd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:28.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.095+0000 7f56f7400640 1 --2- 172.21.15.26:0/828520121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56f8145dc0 0x7f56f81fd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.096+0000 7f56f7400640 1 -- 172.21.15.26:0/828520121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f81fd920 con 0x7f56f8148ea0 2024-04-26T21:10:28.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.096+0000 7f56f6a00640 1 --2- 172.21.15.26:0/828520121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56f8145dc0 0x7f56f81fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:28.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.096+0000 7f56f7400640 1 --2- 172.21.15.26:0/828520121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56f8148ea0 0x7f56f815c490 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f56f000a9c0 tx=0x7f56f0001e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:28.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.096+0000 7f56eea00640 1 -- 172.21.15.26:0/828520121 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f000a4c0 con 0x7f56f8148ea0 2024-04-26T21:10:28.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.096+0000 7f56eea00640 1 -- 172.21.15.26:0/828520121 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56f00019e0 con 0x7f56f8148ea0 2024-04-26T21:10:28.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.096+0000 7f56eea00640 1 -- 172.21.15.26:0/828520121 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0002010 con 0x7f56f8148ea0 2024-04-26T21:10:28.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.096+0000 7f56f7e00640 1 -- 172.21.15.26:0/828520121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f81609b0 con 0x7f56f8148ea0 2024-04-26T21:10:28.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.096+0000 7f56f7e00640 1 -- 172.21.15.26:0/828520121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f815e1c0 con 0x7f56f8148ea0 2024-04-26T21:10:28.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.098+0000 7f56f7e00640 1 -- 172.21.15.26:0/828520121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56f806a700 con 0x7f56f8148ea0 2024-04-26T21:10:28.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.099+0000 7f56eea00640 1 -- 172.21.15.26:0/828520121 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f56f0034a80 con 0x7f56f8148ea0 2024-04-26T21:10:28.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.099+0000 7f56eea00640 1 --2- 172.21.15.26:0/828520121 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f56d806c8f0 0x7f56d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.100+0000 7f56eea00640 1 -- 172.21.15.26:0/828520121 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f56f00abe60 con 0x7f56f8148ea0 2024-04-26T21:10:28.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.100+0000 7f56f6a00640 1 --2- 172.21.15.26:0/828520121 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f56d806c8f0 0x7f56d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.100+0000 7f56f6a00640 1 --2- 172.21.15.26:0/828520121 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f56d806c8f0 0x7f56d806edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009a80 tx=0x7f56e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:28.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.104+0000 7f56eea00640 1 -- 172.21.15.26:0/828520121 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f56f81fd920 con 0x7f56f8148ea0 2024-04-26T21:10:28.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.304+0000 7f56f7e00640 1 -- 172.21.15.26:0/828520121 --> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f56f815d5d0 con 0x7f56d806c8f0 2024-04-26T21:10:28.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.305+0000 7f56eea00640 1 -- 172.21.15.26:0/828520121 <== mgr.4104 v2:172.21.15.26:6832/3130430358 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13495 (secure 0 0 0) 0x7f56f815d5d0 con 0x7f56d806c8f0 2024-04-26T21:10:28.307 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:28.308 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-26T21:10:28.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.308+0000 7f56f7e00640 1 -- 172.21.15.26:0/828520121 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f56d806c8f0 msgr2=0x7f56d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:28.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.308+0000 7f56f7e00640 1 --2- 172.21.15.26:0/828520121 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f56d806c8f0 0x7f56d806edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009a80 tx=0x7f56e8008040 comp rx=0 tx=0).stop 2024-04-26T21:10:28.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.308+0000 7f56f7e00640 1 -- 172.21.15.26:0/828520121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56f8148ea0 msgr2=0x7f56f815c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:28.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.308+0000 7f56f7e00640 1 --2- 172.21.15.26:0/828520121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56f8148ea0 0x7f56f815c490 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f56f000a9c0 tx=0x7f56f0001e30 comp rx=0 tx=0).stop 2024-04-26T21:10:28.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.308+0000 7f56f7e00640 1 -- 172.21.15.26:0/828520121 shutdown_connections 2024-04-26T21:10:28.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.308+0000 7f56f7e00640 1 --2- 172.21.15.26:0/828520121 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f56d806c8f0 0x7f56d806edb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.308+0000 7f56f7e00640 1 --2- 172.21.15.26:0/828520121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56f8148ea0 0x7f56f815c490 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.308+0000 7f56f7e00640 1 --2- 172.21.15.26:0/828520121 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56f81461a0 0x7f56f815bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.308+0000 7f56f7e00640 1 --2- 172.21.15.26:0/828520121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56f8145dc0 0x7f56f81fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.308+0000 7f56f7e00640 1 -- 172.21.15.26:0/828520121 >> 172.21.15.26:0/828520121 conn(0x7f56f806c690 msgr2=0x7f56f80b4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:28.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.308+0000 7f56f7e00640 1 -- 172.21.15.26:0/828520121 shutdown_connections 2024-04-26T21:10:28.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.308+0000 7f56f7e00640 1 -- 172.21.15.26:0/828520121 wait complete. 2024-04-26T21:10:28.327 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2024-04-26T21:10:27.325258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":224,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":237692,"kb_used_data":4856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737028,"statfs":{"total":773094113280,"available":772850716672,"internally_reserved":0,"allocated":4972544,"data_stored":2718830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-59914,"num_objects":-33,"num_object_clones":0,"num_object_copies":-66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-33,"num_whiteouts":0,"num_read":-22,"num_read_kb":-11,"num_write":-192,"num_write_kb":-141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001923"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":207,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T21:10:24.056599+0000","last_change":"2024-04-26T21:08:33.504911+0000","last_active":"2024-04-26T21:10:24.056599+0000","last_peered":"2024-04-26T21:10:24.056599+0000","last_clean":"2024-04-26T21:10:24.056599+0000","last_became_active":"2024-04-26T21:08:33.503667+0000","last_became_peered":"2024-04-26T21:08:33.503667+0000","last_unstale":"2024-04-26T21:10:24.056599+0000","last_undegraded":"2024-04-26T21:10:24.056599+0000","last_fullsized":"2024-04-26T21:10:24.056599+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:32.013907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:32.013907+0000","last_clean_scrub_stamp":"2024-04-26T21:08:32.013907+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:09.596153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1077248,"data_stored":786096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27496,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":483328,"data_stored":195728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":466944,"data_stored":190182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":190182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31636,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":466944,"data_stored":190182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31656,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340184,"statfs":{"total":96636764160,"available":96604348416,"internally_reserved":0,"allocated":483328,"data_stored":195728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":466944,"data_stored":190182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1060864,"data_stored":780550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:10:28.328 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T21:10:28.328 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:10:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.537+0000 7fae5f200640 1 Processor -- start 2024-04-26T21:10:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.538+0000 7fae5f200640 1 -- start start 2024-04-26T21:10:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.539+0000 7fae5f200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae600d2060 0x7fae600d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.539+0000 7fae5f200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae600b61c0 con 0x7fae60162ba0 2024-04-26T21:10:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.539+0000 7fae5f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae600b64a0 con 0x7fae600d1c80 2024-04-26T21:10:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.539+0000 7fae5f200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fae600b5f70 con 0x7fae600d2060 2024-04-26T21:10:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.540+0000 7fae5d400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fae60162ba0 0x7fae600c8b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54638/0 (socket says 172.21.15.26:54638) 2024-04-26T21:10:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.540+0000 7fae5d400640 1 -- 172.21.15.26:0/2351110924 learned_addr learned my addr 172.21.15.26:0/2351110924 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.540+0000 7fae5e800640 1 --2- 172.21.15.26:0/2351110924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae600d2060 0x7fae600d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.540+0000 7fae5ca00640 1 -- 172.21.15.26:0/2351110924 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1995570519 0 0) 0x7fae600b61c0 con 0x7fae60162ba0 2024-04-26T21:10:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.540+0000 7fae5ca00640 1 -- 172.21.15.26:0/2351110924 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae44003520 con 0x7fae60162ba0 2024-04-26T21:10:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.541+0000 7fae5ca00640 1 -- 172.21.15.26:0/2351110924 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3645284221 0 0) 0x7fae44003520 con 0x7fae60162ba0 2024-04-26T21:10:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.541+0000 7fae5ca00640 1 -- 172.21.15.26:0/2351110924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae600d2060 msgr2=0x7fae600d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.541+0000 7fae5ca00640 1 --2- 172.21.15.26:0/2351110924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae600d2060 0x7fae600d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.541+0000 7fae5ca00640 1 -- 172.21.15.26:0/2351110924 >> v1:172.21.15.86:6789/0 conn(0x7fae600d1c80 legacy=0x7fae60162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.541+0000 7fae5ca00640 1 -- 172.21.15.26:0/2351110924 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae60074610 con 0x7fae60162ba0 2024-04-26T21:10:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.541+0000 7fae5ca00640 1 -- 172.21.15.26:0/2351110924 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7fae500019f0 con 0x7fae60162ba0 2024-04-26T21:10:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.541+0000 7fae5ca00640 1 -- 172.21.15.26:0/2351110924 >> v1:172.21.15.26:6789/0 conn(0x7fae60162ba0 legacy=0x7fae600c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.541+0000 7fae5ca00640 1 --2- 172.21.15.26:0/2351110924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae440039e0 0x7fae44003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.541+0000 7fae5ca00640 1 --2- 172.21.15.26:0/2351110924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae600d2060 0x7fae44004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.541+0000 7fae5ca00640 1 --2- 172.21.15.26:0/2351110924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae44004d40 0x7fae440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.541+0000 7fae5ca00640 1 -- 172.21.15.26:0/2351110924 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fae600b5f70 con 0x7fae440039e0 2024-04-26T21:10:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.541+0000 7fae5ca00640 1 -- 172.21.15.26:0/2351110924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae4400bb20 con 0x7fae44004d40 2024-04-26T21:10:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.541+0000 7fae5ca00640 1 -- 172.21.15.26:0/2351110924 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fae4400bd20 con 0x7fae600d2060 2024-04-26T21:10:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.541+0000 7fae5de00640 1 --2- 172.21.15.26:0/2351110924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae440039e0 0x7fae44003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.542+0000 7fae5d400640 1 --2- 172.21.15.26:0/2351110924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae600d2060 0x7fae44004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.542+0000 7fae5e800640 1 --2- 172.21.15.26:0/2351110924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae44004d40 0x7fae440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.542+0000 7fae5de00640 1 -- 172.21.15.26:0/2351110924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae600d2060 msgr2=0x7fae44004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:28.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.542+0000 7fae5de00640 1 --2- 172.21.15.26:0/2351110924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae600d2060 0x7fae44004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.542+0000 7fae5de00640 1 -- 172.21.15.26:0/2351110924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae44004d40 msgr2=0x7fae440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:28.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.542+0000 7fae5de00640 1 --2- 172.21.15.26:0/2351110924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae44004d40 0x7fae440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.542+0000 7fae5de00640 1 -- 172.21.15.26:0/2351110924 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae44003520 con 0x7fae440039e0 2024-04-26T21:10:28.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.542+0000 7fae5e800640 1 --2- 172.21.15.26:0/2351110924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae44004d40 0x7fae440051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:10:28.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.543+0000 7fae5de00640 1 --2- 172.21.15.26:0/2351110924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae440039e0 0x7fae44003e00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fae48001750 tx=0x7fae44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88bda46011f0fb8e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:28.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.543+0000 7fae5ca00640 1 -- 172.21.15.26:0/2351110924 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48001370 con 0x7fae440039e0 2024-04-26T21:10:28.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.543+0000 7fae5ca00640 1 -- 172.21.15.26:0/2351110924 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae480052d0 con 0x7fae440039e0 2024-04-26T21:10:28.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.543+0000 7fae5ca00640 1 -- 172.21.15.26:0/2351110924 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4802ab20 con 0x7fae440039e0 2024-04-26T21:10:28.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.544+0000 7fae5f200640 1 -- 172.21.15.26:0/2351110924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae440039e0 msgr2=0x7fae44003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:28.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.544+0000 7fae5f200640 1 --2- 172.21.15.26:0/2351110924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae440039e0 0x7fae44003e00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fae48001750 tx=0x7fae44005830 comp rx=0 tx=0).stop 2024-04-26T21:10:28.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.544+0000 7fae5e800640 1 -- 172.21.15.26:0/2351110924 reap_dead start 2024-04-26T21:10:28.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.544+0000 7fae5f200640 1 -- 172.21.15.26:0/2351110924 shutdown_connections 2024-04-26T21:10:28.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.544+0000 7fae5f200640 1 -- 172.21.15.26:0/2351110924 >> 172.21.15.26:0/2351110924 conn(0x7fae600d5020 msgr2=0x7fae600d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:28.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.544+0000 7fae5f200640 1 -- 172.21.15.26:0/2351110924 shutdown_connections 2024-04-26T21:10:28.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.544+0000 7fae5f200640 1 -- 172.21.15.26:0/2351110924 wait complete. 2024-04-26T21:10:28.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.546+0000 7fae5f200640 1 Processor -- start 2024-04-26T21:10:28.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.546+0000 7fae5f200640 1 -- start start 2024-04-26T21:10:28.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.547+0000 7fae5f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae440039e0 0x7fae601538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.547+0000 7fae5f200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae600cc0f0 0x7fae600cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.547+0000 7fae5f200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae6006d200 0x7fae6006d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.547+0000 7fae5f200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fae600d3830 con 0x7fae6006d200 2024-04-26T21:10:28.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.547+0000 7fae5f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae600d5d50 con 0x7fae440039e0 2024-04-26T21:10:28.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.547+0000 7fae5f200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fae600d0cf0 con 0x7fae600cc0f0 2024-04-26T21:10:28.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.547+0000 7fae5e800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae6006d200 0x7fae6006d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.547+0000 7fae5e800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae6006d200 0x7fae6006d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57286/0 (socket says 172.21.15.26:57286) 2024-04-26T21:10:28.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.547+0000 7fae5e800640 1 -- 172.21.15.26:0/3675741142 learned_addr learned my addr 172.21.15.26:0/3675741142 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:28.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.548+0000 7fae5e800640 1 -- 172.21.15.26:0/3675741142 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae600cc0f0 msgr2=0x7fae600cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:28.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.548+0000 7fae5e800640 1 --2- 172.21.15.26:0/3675741142 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae600cc0f0 0x7fae600cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.548+0000 7fae5e800640 1 -- 172.21.15.26:0/3675741142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae440039e0 msgr2=0x7fae601538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:28.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.548+0000 7fae5de00640 1 --2- 172.21.15.26:0/3675741142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae440039e0 0x7fae601538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.548+0000 7fae5e800640 1 --2- 172.21.15.26:0/3675741142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae440039e0 0x7fae601538c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.548+0000 7fae5e800640 1 -- 172.21.15.26:0/3675741142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae600ccc20 con 0x7fae6006d200 2024-04-26T21:10:28.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.548+0000 7fae5e800640 1 --2- 172.21.15.26:0/3675741142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae6006d200 0x7fae6006d680 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fae600d2510 tx=0x7fae58011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:28.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.548+0000 7fae56000640 1 -- 172.21.15.26:0/3675741142 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58003610 con 0x7fae6006d200 2024-04-26T21:10:28.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.548+0000 7fae56000640 1 -- 172.21.15.26:0/3675741142 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae5801b070 con 0x7fae6006d200 2024-04-26T21:10:28.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.548+0000 7fae56000640 1 -- 172.21.15.26:0/3675741142 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae580039f0 con 0x7fae6006d200 2024-04-26T21:10:28.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.548+0000 7fae5f200640 1 -- 172.21.15.26:0/3675741142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae600ccf00 con 0x7fae6006d200 2024-04-26T21:10:28.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.548+0000 7fae5f200640 1 -- 172.21.15.26:0/3675741142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae600b8090 con 0x7fae6006d200 2024-04-26T21:10:28.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.550+0000 7fae5f200640 1 -- 172.21.15.26:0/3675741142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae600b8290 con 0x7fae6006d200 2024-04-26T21:10:28.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.551+0000 7fae56000640 1 -- 172.21.15.26:0/3675741142 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae58003b90 con 0x7fae6006d200 2024-04-26T21:10:28.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.552+0000 7fae56000640 1 --2- 172.21.15.26:0/3675741142 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fae3c06c8f0 0x7fae3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.552+0000 7fae56000640 1 -- 172.21.15.26:0/3675741142 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7fae5808b920 con 0x7fae6006d200 2024-04-26T21:10:28.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.552+0000 7fae5de00640 1 --2- 172.21.15.26:0/3675741142 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fae3c06c8f0 0x7fae3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.553+0000 7fae5de00640 1 --2- 172.21.15.26:0/3675741142 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fae3c06c8f0 0x7fae3c06edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fae48032000 tx=0x7fae4802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:28.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.556+0000 7fae56000640 1 -- 172.21.15.26:0/3675741142 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fae600b8290 con 0x7fae6006d200 2024-04-26T21:10:28.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.753+0000 7fae5f200640 1 -- 172.21.15.26:0/3675741142 --> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fae6006a700 con 0x7fae3c06c8f0 2024-04-26T21:10:28.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.755+0000 7fae56000640 1 -- 172.21.15.26:0/3675741142 <== mgr.4104 v2:172.21.15.26:6832/3130430358 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13495 (secure 0 0 0) 0x7fae6006a700 con 0x7fae3c06c8f0 2024-04-26T21:10:28.757 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:28.757 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-26T21:10:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.758+0000 7fae5f200640 1 -- 172.21.15.26:0/3675741142 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fae3c06c8f0 msgr2=0x7fae3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.758+0000 7fae5f200640 1 --2- 172.21.15.26:0/3675741142 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fae3c06c8f0 0x7fae3c06edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fae48032000 tx=0x7fae4802e000 comp rx=0 tx=0).stop 2024-04-26T21:10:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.758+0000 7fae5f200640 1 -- 172.21.15.26:0/3675741142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae6006d200 msgr2=0x7fae6006d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.758+0000 7fae5f200640 1 --2- 172.21.15.26:0/3675741142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae6006d200 0x7fae6006d680 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fae600d2510 tx=0x7fae58011920 comp rx=0 tx=0).stop 2024-04-26T21:10:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.758+0000 7fae5f200640 1 -- 172.21.15.26:0/3675741142 shutdown_connections 2024-04-26T21:10:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.758+0000 7fae5f200640 1 --2- 172.21.15.26:0/3675741142 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fae3c06c8f0 0x7fae3c06edb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.758+0000 7fae5f200640 1 --2- 172.21.15.26:0/3675741142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae6006d200 0x7fae6006d680 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.758+0000 7fae5f200640 1 --2- 172.21.15.26:0/3675741142 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae600cc0f0 0x7fae600cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.758+0000 7fae5f200640 1 --2- 172.21.15.26:0/3675741142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae440039e0 0x7fae601538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.758+0000 7fae5f200640 1 -- 172.21.15.26:0/3675741142 >> 172.21.15.26:0/3675741142 conn(0x7fae600d5020 msgr2=0x7fae60160130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:28.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.758+0000 7fae5f200640 1 -- 172.21.15.26:0/3675741142 shutdown_connections 2024-04-26T21:10:28.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.758+0000 7fae5f200640 1 -- 172.21.15.26:0/3675741142 wait complete. 2024-04-26T21:10:28.777 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2024-04-26T21:10:27.325258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":224,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":237692,"kb_used_data":4856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737028,"statfs":{"total":773094113280,"available":772850716672,"internally_reserved":0,"allocated":4972544,"data_stored":2718830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-59914,"num_objects":-33,"num_object_clones":0,"num_object_copies":-66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-33,"num_whiteouts":0,"num_read":-22,"num_read_kb":-11,"num_write":-192,"num_write_kb":-141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001923"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":207,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T21:10:24.056599+0000","last_change":"2024-04-26T21:08:33.504911+0000","last_active":"2024-04-26T21:10:24.056599+0000","last_peered":"2024-04-26T21:10:24.056599+0000","last_clean":"2024-04-26T21:10:24.056599+0000","last_became_active":"2024-04-26T21:08:33.503667+0000","last_became_peered":"2024-04-26T21:08:33.503667+0000","last_unstale":"2024-04-26T21:10:24.056599+0000","last_undegraded":"2024-04-26T21:10:24.056599+0000","last_fullsized":"2024-04-26T21:10:24.056599+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:32.013907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:32.013907+0000","last_clean_scrub_stamp":"2024-04-26T21:08:32.013907+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:09.596153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1077248,"data_stored":786096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27496,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":483328,"data_stored":195728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":466944,"data_stored":190182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":190182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31636,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":466944,"data_stored":190182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31656,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340184,"statfs":{"total":96636764160,"available":96604348416,"internally_reserved":0,"allocated":483328,"data_stored":195728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":466944,"data_stored":190182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1060864,"data_stored":780550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:10:28.777 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:10:28.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.984+0000 7f1001800640 1 Processor -- start 2024-04-26T21:10:28.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.984+0000 7f1001800640 1 -- start start 2024-04-26T21:10:28.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.985+0000 7f1001800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0ffc146990 0x7f0ffc0b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.985+0000 7f1001800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ffc06ec20 0x7f0ffc0d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.985+0000 7f1001800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ffc0b4480 con 0x7f0ffc0d1210 2024-04-26T21:10:28.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.985+0000 7f1001800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f0ffc146d70 con 0x7f0ffc146990 2024-04-26T21:10:28.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.985+0000 7f1001800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0ffc0d1640 con 0x7f0ffc06ec20 2024-04-26T21:10:28.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.985+0000 7f0ffaa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0ffc146990 0x7f0ffc0b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.986+0000 7f0ffaa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0ffc146990 0x7f0ffc0b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48370/0 (socket says 172.21.15.26:48370) 2024-04-26T21:10:28.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.986+0000 7f0ffaa00640 1 -- 172.21.15.26:0/2572878571 learned_addr learned my addr 172.21.15.26:0/2572878571 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:28.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.986+0000 7f0ffa000640 1 --2- 172.21.15.26:0/2572878571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ffc06ec20 0x7f0ffc0d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.986+0000 7f0ffaa00640 1 -- 172.21.15.26:0/2572878571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ffc06ec20 msgr2=0x7f0ffc0d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:28.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.986+0000 7f0ffaa00640 1 --2- 172.21.15.26:0/2572878571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ffc06ec20 0x7f0ffc0d0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.986+0000 7f0ffaa00640 1 -- 172.21.15.26:0/2572878571 >> v1:172.21.15.86:6789/0 conn(0x7f0ffc0d1210 legacy=0x7f0ffc072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:28.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.986+0000 7f0ffaa00640 1 -- 172.21.15.26:0/2572878571 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc073b70 con 0x7f0ffc146990 2024-04-26T21:10:28.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.987+0000 7f0ffaa00640 1 --2- 172.21.15.26:0/2572878571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ffc146990 0x7f0ffc0b3f40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009870 tx=0x7f0fec02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7e94de58c593bb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:28.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.987+0000 7f0ff9600640 1 -- 172.21.15.26:0/2572878571 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec037070 con 0x7f0ffc146990 2024-04-26T21:10:28.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.987+0000 7f0ff9600640 1 -- 172.21.15.26:0/2572878571 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fec032440 con 0x7f0ffc146990 2024-04-26T21:10:28.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.987+0000 7f0ff9600640 1 -- 172.21.15.26:0/2572878571 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec032810 con 0x7f0ffc146990 2024-04-26T21:10:28.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.988+0000 7f1001800640 1 -- 172.21.15.26:0/2572878571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ffc146990 msgr2=0x7f0ffc0b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:28.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.988+0000 7f1001800640 1 --2- 172.21.15.26:0/2572878571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ffc146990 0x7f0ffc0b3f40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009870 tx=0x7f0fec02a210 comp rx=0 tx=0).stop 2024-04-26T21:10:28.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.988+0000 7f1001800640 1 -- 172.21.15.26:0/2572878571 shutdown_connections 2024-04-26T21:10:28.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.988+0000 7f1001800640 1 --2- 172.21.15.26:0/2572878571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ffc06ec20 0x7f0ffc0d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.988+0000 7f1001800640 1 --2- 172.21.15.26:0/2572878571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ffc146990 0x7f0ffc0b3f40 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.988+0000 7f1001800640 1 -- 172.21.15.26:0/2572878571 >> 172.21.15.26:0/2572878571 conn(0x7f0ffc0cd840 msgr2=0x7f0ffc0cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:28.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.988+0000 7f1001800640 1 -- 172.21.15.26:0/2572878571 shutdown_connections 2024-04-26T21:10:28.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.988+0000 7f1001800640 1 -- 172.21.15.26:0/2572878571 wait complete. 2024-04-26T21:10:28.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.990+0000 7f1001800640 1 Processor -- start 2024-04-26T21:10:28.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.990+0000 7f1001800640 1 -- start start 2024-04-26T21:10:28.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.991+0000 7f1001800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ffc06ec20 0x7f0ffc1439a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.991+0000 7f1001800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ffc0d1210 0x7f0ffc13e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.991+0000 7f1001800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ffc146990 0x7f0ffc140c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.991+0000 7f1001800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc06e420 con 0x7f0ffc146990 2024-04-26T21:10:28.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.991+0000 7f1001800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc074130 con 0x7f0ffc0d1210 2024-04-26T21:10:28.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.991+0000 7f1001800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0ffc06e700 con 0x7f0ffc06ec20 2024-04-26T21:10:28.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.991+0000 7f0ffb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ffc146990 0x7f0ffc140c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.991+0000 7f0ffb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ffc146990 0x7f0ffc140c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48386/0 (socket says 172.21.15.26:48386) 2024-04-26T21:10:28.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.991+0000 7f0ffb400640 1 -- 172.21.15.26:0/687010990 learned_addr learned my addr 172.21.15.26:0/687010990 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:28.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.992+0000 7f0ffa000640 1 --2- 172.21.15.26:0/687010990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ffc0d1210 0x7f0ffc13e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.992+0000 7f0ffaa00640 1 --2- 172.21.15.26:0/687010990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ffc06ec20 0x7f0ffc1439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.992+0000 7f0ffb400640 1 -- 172.21.15.26:0/687010990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ffc06ec20 msgr2=0x7f0ffc1439a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:28.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.992+0000 7f0ffb400640 1 --2- 172.21.15.26:0/687010990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ffc06ec20 0x7f0ffc1439a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.992+0000 7f0ffb400640 1 -- 172.21.15.26:0/687010990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ffc0d1210 msgr2=0x7f0ffc13e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:28.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.992+0000 7f0ffb400640 1 --2- 172.21.15.26:0/687010990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ffc0d1210 0x7f0ffc13e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:28.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.992+0000 7f0ffb400640 1 -- 172.21.15.26:0/687010990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc142900 con 0x7f0ffc146990 2024-04-26T21:10:28.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.992+0000 7f0ffaa00640 1 --2- 172.21.15.26:0/687010990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ffc06ec20 0x7f0ffc1439a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:28.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.992+0000 7f0ffb400640 1 --2- 172.21.15.26:0/687010990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ffc146990 0x7f0ffc140c90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0ff400b970 tx=0x7f0ff400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:28.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.992+0000 7f0ff2a00640 1 -- 172.21.15.26:0/687010990 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4019070 con 0x7f0ffc146990 2024-04-26T21:10:28.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.993+0000 7f0ff2a00640 1 -- 172.21.15.26:0/687010990 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ff400cae0 con 0x7f0ffc146990 2024-04-26T21:10:28.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.993+0000 7f0ff2a00640 1 -- 172.21.15.26:0/687010990 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4014440 con 0x7f0ffc146990 2024-04-26T21:10:28.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.993+0000 7f1001800640 1 -- 172.21.15.26:0/687010990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ffc142be0 con 0x7f0ffc146990 2024-04-26T21:10:28.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.993+0000 7f1001800640 1 -- 172.21.15.26:0/687010990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ffc1431a0 con 0x7f0ffc146990 2024-04-26T21:10:28.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.994+0000 7f1001800640 1 -- 172.21.15.26:0/687010990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ffc06a700 con 0x7f0ffc146990 2024-04-26T21:10:28.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.995+0000 7f0ff2a00640 1 -- 172.21.15.26:0/687010990 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ff401c070 con 0x7f0ffc146990 2024-04-26T21:10:28.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.996+0000 7f0ff2a00640 1 --2- 172.21.15.26:0/687010990 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0fdc06c940 0x7f0fdc06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:28.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.997+0000 7f0ff2a00640 1 -- 172.21.15.26:0/687010990 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f0ff408b4e0 con 0x7f0ffc146990 2024-04-26T21:10:28.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.997+0000 7f0ffaa00640 1 --2- 172.21.15.26:0/687010990 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0fdc06c940 0x7f0fdc06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:28.997+0000 7f0ffaa00640 1 --2- 172.21.15.26:0/687010990 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0fdc06c940 0x7f0fdc06ee00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0fec034000 tx=0x7f0fec02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:29.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:29.001+0000 7f0ff2a00640 1 -- 172.21.15.26:0/687010990 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0ff4059c80 con 0x7f0ffc146990 2024-04-26T21:10:29.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:29.209+0000 7f1001800640 1 -- 172.21.15.26:0/687010990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0ffc06fd50 con 0x7f0ffc146990 2024-04-26T21:10:29.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:29.211+0000 7f0ff2a00640 1 -- 172.21.15.26:0/687010990 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+12972 (secure 0 0 0) 0x7f0ff4059580 con 0x7f0ffc146990 2024-04-26T21:10:29.213 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:29.213 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":41,"fsid":"eaf60df9-83a0-4283-ab2c-83d2c02a2067","created":"2024-04-26T21:08:18.113621+0000","modified":"2024-04-26T21:10:24.039336+0000","last_up_change":"2024-04-26T21:08:30.006391+0000","last_in_change":"2024-04-26T21:08:21.519694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:08:31.322511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80aba520-1a3a-455f-9941-72e6bbfdb5f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6811","nonce":2924039897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6813","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6815","nonce":2924039897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6823","nonce":2924039897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":2924039897},{"type":"v1","addr":"172.21.15.26:6820","nonce":2924039897}]},"public_addr":"172.21.15.26:6811/2924039897","cluster_addr":"172.21.15.26:6815/2924039897","heartbeat_back_addr":"172.21.15.26:6823/2924039897","heartbeat_front_addr":"172.21.15.26:6820/2924039897","state":["exists","up"]},{"osd":1,"uuid":"3084f542-c355-437c-8db5-6164ebc4c5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6801","nonce":398686833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6803","nonce":398686833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6807","nonce":398686833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":398686833},{"type":"v1","addr":"172.21.15.26:6805","nonce":398686833}]},"public_addr":"172.21.15.26:6801/398686833","cluster_addr":"172.21.15.26:6803/398686833","heartbeat_back_addr":"172.21.15.26:6807/398686833","heartbeat_front_addr":"172.21.15.26:6805/398686833","state":["exists","up"]},{"osd":2,"uuid":"edbc1aa5-f21f-4521-a569-4e90e1384395","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6825","nonce":2316564464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6827","nonce":2316564464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6831","nonce":2316564464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":2316564464},{"type":"v1","addr":"172.21.15.26:6829","nonce":2316564464}]},"public_addr":"172.21.15.26:6825/2316564464","cluster_addr":"172.21.15.26:6827/2316564464","heartbeat_back_addr":"172.21.15.26:6831/2316564464","heartbeat_front_addr":"172.21.15.26:6829/2316564464","state":["exists","up"]},{"osd":3,"uuid":"2bbd1d2f-aeb3-42cb-bf45-041def845223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6810","nonce":1280758061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6814","nonce":1280758061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6819","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6821","nonce":1280758061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1280758061},{"type":"v1","addr":"172.21.15.26:6817","nonce":1280758061}]},"public_addr":"172.21.15.26:6810/1280758061","cluster_addr":"172.21.15.26:6814/1280758061","heartbeat_back_addr":"172.21.15.26:6821/1280758061","heartbeat_front_addr":"172.21.15.26:6817/1280758061","state":["exists","up"]},{"osd":4,"uuid":"b4fc5b2a-49e1-4ba3-b7c9-9426e78a1698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6817","nonce":1450967413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6819","nonce":1450967413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6824","nonce":1450967413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1450967413},{"type":"v1","addr":"172.21.15.86:6821","nonce":1450967413}]},"public_addr":"172.21.15.86:6817/1450967413","cluster_addr":"172.21.15.86:6819/1450967413","heartbeat_back_addr":"172.21.15.86:6824/1450967413","heartbeat_front_addr":"172.21.15.86:6821/1450967413","state":["exists","up"]},{"osd":5,"uuid":"6320cdc6-43d3-444f-95dc-a3ddc69cf718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6801","nonce":1459341072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6803","nonce":1459341072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6807","nonce":1459341072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":1459341072},{"type":"v1","addr":"172.21.15.86:6805","nonce":1459341072}]},"public_addr":"172.21.15.86:6801/1459341072","cluster_addr":"172.21.15.86:6803/1459341072","heartbeat_back_addr":"172.21.15.86:6807/1459341072","heartbeat_front_addr":"172.21.15.86:6805/1459341072","state":["exists","up"]},{"osd":6,"uuid":"be31e84f-e95f-4bd5-ae28-1d9703d8c09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6823","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6825","nonce":4235789402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6827","nonce":4235789402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6831","nonce":4235789402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":4235789402},{"type":"v1","addr":"172.21.15.86:6829","nonce":4235789402}]},"public_addr":"172.21.15.86:6825/4235789402","cluster_addr":"172.21.15.86:6827/4235789402","heartbeat_back_addr":"172.21.15.86:6831/4235789402","heartbeat_front_addr":"172.21.15.86:6829/4235789402","state":["exists","up"]},{"osd":7,"uuid":"5e704415-8a84-4f70-92cc-e5b34395772f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6809","nonce":2582257935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6811","nonce":2582257935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6815","nonce":2582257935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2582257935},{"type":"v1","addr":"172.21.15.86:6813","nonce":2582257935}]},"public_addr":"172.21.15.86:6809/2582257935","cluster_addr":"172.21.15.86:6811/2582257935","heartbeat_back_addr":"172.21.15.86:6815/2582257935","heartbeat_front_addr":"172.21.15.86:6813/2582257935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.869541+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.854153+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.742131+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.920282+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.763351+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.908429+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.965985+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:08:26.930189+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6836/2528180854":"2024-04-27T21:10:20.006597+0000","172.21.15.26:6835/2528180854":"2024-04-27T21:10:20.006597+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:10:29.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:29.214+0000 7f1001800640 1 -- 172.21.15.26:0/687010990 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0fdc06c940 msgr2=0x7f0fdc06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:29.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:29.214+0000 7f1001800640 1 --2- 172.21.15.26:0/687010990 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0fdc06c940 0x7f0fdc06ee00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0fec034000 tx=0x7f0fec02e000 comp rx=0 tx=0).stop 2024-04-26T21:10:29.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:29.214+0000 7f1001800640 1 -- 172.21.15.26:0/687010990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ffc146990 msgr2=0x7f0ffc140c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:29.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:29.214+0000 7f1001800640 1 --2- 172.21.15.26:0/687010990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ffc146990 0x7f0ffc140c90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0ff400b970 tx=0x7f0ff400be30 comp rx=0 tx=0).stop 2024-04-26T21:10:29.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:29.214+0000 7f1001800640 1 -- 172.21.15.26:0/687010990 shutdown_connections 2024-04-26T21:10:29.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:29.214+0000 7f1001800640 1 --2- 172.21.15.26:0/687010990 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f0fdc06c940 0x7f0fdc06ee00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:29.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:29.214+0000 7f1001800640 1 --2- 172.21.15.26:0/687010990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ffc146990 0x7f0ffc140c90 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:29.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:29.214+0000 7f1001800640 1 --2- 172.21.15.26:0/687010990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ffc0d1210 0x7f0ffc13e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:29.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:29.214+0000 7f1001800640 1 --2- 172.21.15.26:0/687010990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ffc06ec20 0x7f0ffc1439a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:29.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:29.214+0000 7f1001800640 1 -- 172.21.15.26:0/687010990 >> 172.21.15.26:0/687010990 conn(0x7f0ffc0cd840 msgr2=0x7f0ffc15d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:29.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:29.214+0000 7f1001800640 1 -- 172.21.15.26:0/687010990 shutdown_connections 2024-04-26T21:10:29.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:29.214+0000 7f1001800640 1 -- 172.21.15.26:0/687010990 wait complete. 2024-04-26T21:10:30.237 INFO:tasks.ceph:Scrubbing osd.0 2024-04-26T21:10:30.238 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:10:30.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.465+0000 7f57fcc00640 1 Processor -- start 2024-04-26T21:10:30.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.466+0000 7f57fcc00640 1 -- start start 2024-04-26T21:10:30.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.467+0000 7f57fcc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f8162b40 0x7f57f80b7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:30.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.467+0000 7f57fcc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f57f80d1a50 0x7f57f80d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:30.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.468+0000 7f57fcc00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57f80b66a0 con 0x7f57f8162760 2024-04-26T21:10:30.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.468+0000 7f57fcc00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f57f80d4080 con 0x7f57f80d1a50 2024-04-26T21:10:30.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.468+0000 7f57fcc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f57f80cfc00 con 0x7f57f8162b40 2024-04-26T21:10:30.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.468+0000 7f57f7400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f57f8162760 0x7f57f80b5f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33428/0 (socket says 172.21.15.26:33428) 2024-04-26T21:10:30.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.468+0000 7f57f7400640 1 -- 172.21.15.26:0/1969880192 learned_addr learned my addr 172.21.15.26:0/1969880192 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:30.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.468+0000 7f57f7e00640 1 --2- 172.21.15.26:0/1969880192 >> v2:172.21.15.26:3300/0 conn(0x7f57f80d1a50 0x7f57f80d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:30.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.468+0000 7f57f6a00640 1 --2- 172.21.15.26:0/1969880192 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f8162b40 0x7f57f80b7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:30.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.469+0000 7f57f6000640 1 -- 172.21.15.26:0/1969880192 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3916995603 0 0) 0x7f57f80b66a0 con 0x7f57f8162760 2024-04-26T21:10:30.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.469+0000 7f57f6000640 1 -- 172.21.15.26:0/1969880192 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57dc003520 con 0x7f57f8162760 2024-04-26T21:10:30.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.469+0000 7f57f7e00640 1 -- 172.21.15.26:0/1969880192 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f8162b40 msgr2=0x7f57f80b7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:30.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.469+0000 7f57f7e00640 1 --2- 172.21.15.26:0/1969880192 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f8162b40 0x7f57f80b7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:30.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.469+0000 7f57f7e00640 1 -- 172.21.15.26:0/1969880192 >> v1:172.21.15.26:6789/0 conn(0x7f57f8162760 legacy=0x7f57f80b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:30.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.469+0000 7f57f7e00640 1 -- 172.21.15.26:0/1969880192 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f80d37e0 con 0x7f57f80d1a50 2024-04-26T21:10:30.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.470+0000 7f57f7e00640 1 --2- 172.21.15.26:0/1969880192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57f80d1a50 0x7f57f80d1ee0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f57f00097a0 tx=0x7f57f002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2f6ef8cb829f43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:30.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.470+0000 7f57f6000640 1 -- 172.21.15.26:0/1969880192 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0037070 con 0x7f57f80d1a50 2024-04-26T21:10:30.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.470+0000 7f57f6000640 1 -- 172.21.15.26:0/1969880192 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f002ae50 con 0x7f57f80d1a50 2024-04-26T21:10:30.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.470+0000 7f57f6000640 1 -- 172.21.15.26:0/1969880192 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0032660 con 0x7f57f80d1a50 2024-04-26T21:10:30.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.471+0000 7f57fcc00640 1 -- 172.21.15.26:0/1969880192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57f80d1a50 msgr2=0x7f57f80d1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:30.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.471+0000 7f57fcc00640 1 --2- 172.21.15.26:0/1969880192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57f80d1a50 0x7f57f80d1ee0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f57f00097a0 tx=0x7f57f002a100 comp rx=0 tx=0).stop 2024-04-26T21:10:30.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.471+0000 7f57fcc00640 1 -- 172.21.15.26:0/1969880192 shutdown_connections 2024-04-26T21:10:30.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.471+0000 7f57fcc00640 1 --2- 172.21.15.26:0/1969880192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57f80d1a50 0x7f57f80d1ee0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:30.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.471+0000 7f57fcc00640 1 --2- 172.21.15.26:0/1969880192 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f8162b40 0x7f57f80b7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:30.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.471+0000 7f57fcc00640 1 -- 172.21.15.26:0/1969880192 >> 172.21.15.26:0/1969880192 conn(0x7f57f80d5060 msgr2=0x7f57f80d5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:30.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.471+0000 7f57fcc00640 1 -- 172.21.15.26:0/1969880192 shutdown_connections 2024-04-26T21:10:30.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.471+0000 7f57fcc00640 1 -- 172.21.15.26:0/1969880192 wait complete. 2024-04-26T21:10:30.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.473+0000 7f57fcc00640 1 Processor -- start 2024-04-26T21:10:30.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.473+0000 7f57fcc00640 1 -- start start 2024-04-26T21:10:30.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.474+0000 7f57fcc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57f80d1a50 0x7f57f815b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:30.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.474+0000 7f57fcc00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f8162760 0x7f57f815c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:30.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.474+0000 7f57fcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f8162b40 0x7f57f80b8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:30.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.474+0000 7f57fcc00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f57f80cfea0 con 0x7f57f80d1a50 2024-04-26T21:10:30.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.474+0000 7f57fcc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57f80d3da0 con 0x7f57f8162b40 2024-04-26T21:10:30.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.474+0000 7f57fcc00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f57f80d5d50 con 0x7f57f8162760 2024-04-26T21:10:30.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.474+0000 7f57f7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57f80d1a50 0x7f57f815b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:30.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.474+0000 7f57f7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57f80d1a50 0x7f57f815b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48410/0 (socket says 172.21.15.26:48410) 2024-04-26T21:10:30.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.474+0000 7f57f7400640 1 -- 172.21.15.26:0/2848619399 learned_addr learned my addr 172.21.15.26:0/2848619399 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:30.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.475+0000 7f57f6a00640 1 --2- 172.21.15.26:0/2848619399 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f8162760 0x7f57f815c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:30.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.475+0000 7f57f7e00640 1 --2- 172.21.15.26:0/2848619399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f8162b40 0x7f57f80b8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:30.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.475+0000 7f57f7400640 1 -- 172.21.15.26:0/2848619399 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f8162760 msgr2=0x7f57f815c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:30.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.475+0000 7f57f7400640 1 --2- 172.21.15.26:0/2848619399 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f8162760 0x7f57f815c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:30.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.475+0000 7f57f7400640 1 -- 172.21.15.26:0/2848619399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f8162b40 msgr2=0x7f57f80b8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:30.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.475+0000 7f57f7400640 1 --2- 172.21.15.26:0/2848619399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f8162b40 0x7f57f80b8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:30.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.475+0000 7f57f7400640 1 -- 172.21.15.26:0/2848619399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f80b47e0 con 0x7f57f80d1a50 2024-04-26T21:10:30.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.475+0000 7f57f7400640 1 --2- 172.21.15.26:0/2848619399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57f80d1a50 0x7f57f815b560 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f57e800cb70 tx=0x7f57e800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:30.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.476+0000 7f57ef400640 1 -- 172.21.15.26:0/2848619399 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e8017070 con 0x7f57f80d1a50 2024-04-26T21:10:30.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.476+0000 7f57ef400640 1 -- 172.21.15.26:0/2848619399 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57e800ad40 con 0x7f57f80d1a50 2024-04-26T21:10:30.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.476+0000 7f57ef400640 1 -- 172.21.15.26:0/2848619399 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e8012620 con 0x7f57f80d1a50 2024-04-26T21:10:30.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.476+0000 7f57fcc00640 1 -- 172.21.15.26:0/2848619399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f80b4ac0 con 0x7f57f80d1a50 2024-04-26T21:10:30.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.476+0000 7f57fcc00640 1 -- 172.21.15.26:0/2848619399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f80b5080 con 0x7f57f80d1a50 2024-04-26T21:10:30.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.477+0000 7f57fcc00640 1 -- 172.21.15.26:0/2848619399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f57f806a700 con 0x7f57f80d1a50 2024-04-26T21:10:30.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.478+0000 7f57ef400640 1 -- 172.21.15.26:0/2848619399 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57e801a070 con 0x7f57f80d1a50 2024-04-26T21:10:30.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.479+0000 7f57ef400640 1 --2- 172.21.15.26:0/2848619399 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f57d406c8f0 0x7f57d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:30.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.479+0000 7f57ef400640 1 -- 172.21.15.26:0/2848619399 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f57e808b780 con 0x7f57f80d1a50 2024-04-26T21:10:30.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.479+0000 7f57f6a00640 1 --2- 172.21.15.26:0/2848619399 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f57d406c8f0 0x7f57d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:30.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.479+0000 7f57ef400640 1 --2- 172.21.15.26:0/2848619399 >> [v2:172.21.15.26:6808/2924039897,v1:172.21.15.26:6811/2924039897] conn(0x7f57d40727b0 0x7f57d4074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:30.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.479+0000 7f57ef400640 1 -- 172.21.15.26:0/2848619399 --> [v2:172.21.15.26:6808/2924039897,v1:172.21.15.26:6811/2924039897] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f57d4075280 con 0x7f57d40727b0 2024-04-26T21:10:30.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.479+0000 7f57ef400640 1 -- 172.21.15.26:0/2848619399 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f57e8092050 con 0x7f57f80d1a50 2024-04-26T21:10:30.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.479+0000 7f57f7e00640 1 --2- 172.21.15.26:0/2848619399 >> [v2:172.21.15.26:6808/2924039897,v1:172.21.15.26:6811/2924039897] conn(0x7f57d40727b0 0x7f57d4074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:30.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.480+0000 7f57f6a00640 1 --2- 172.21.15.26:0/2848619399 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f57d406c8f0 0x7f57d406edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f57f815f240 tx=0x7f57e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:30.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.480+0000 7f57f7e00640 1 --2- 172.21.15.26:0/2848619399 >> [v2:172.21.15.26:6808/2924039897,v1:172.21.15.26:6811/2924039897] conn(0x7f57d40727b0 0x7f57d4074bd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:30.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.481+0000 7f57ef400640 1 -- 172.21.15.26:0/2848619399 <== osd.0 v2:172.21.15.26:6808/2924039897 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57d4075280 con 0x7f57d40727b0 2024-04-26T21:10:30.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.496+0000 7f57fcc00640 1 -- 172.21.15.26:0/2848619399 --> [v2:172.21.15.26:6808/2924039897,v1:172.21.15.26:6811/2924039897] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f57f808cf70 con 0x7f57d40727b0 2024-04-26T21:10:30.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.496+0000 7f57ef400640 1 -- 172.21.15.26:0/2848619399 <== osd.0 v2:172.21.15.26:6808/2924039897 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f57f808cf70 con 0x7f57d40727b0 2024-04-26T21:10:30.498 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-26T21:10:30.498 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:10:30.498 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-26T21:10:30.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.497+0000 7f57fcc00640 1 -- 172.21.15.26:0/2848619399 >> [v2:172.21.15.26:6808/2924039897,v1:172.21.15.26:6811/2924039897] conn(0x7f57d40727b0 msgr2=0x7f57d4074bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:30.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.497+0000 7f57fcc00640 1 --2- 172.21.15.26:0/2848619399 >> [v2:172.21.15.26:6808/2924039897,v1:172.21.15.26:6811/2924039897] conn(0x7f57d40727b0 0x7f57d4074bd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:30.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.497+0000 7f57fcc00640 1 -- 172.21.15.26:0/2848619399 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f57d406c8f0 msgr2=0x7f57d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:30.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.497+0000 7f57fcc00640 1 --2- 172.21.15.26:0/2848619399 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f57d406c8f0 0x7f57d406edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f57f815f240 tx=0x7f57e0008040 comp rx=0 tx=0).stop 2024-04-26T21:10:30.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.497+0000 7f57fcc00640 1 -- 172.21.15.26:0/2848619399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57f80d1a50 msgr2=0x7f57f815b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:30.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.497+0000 7f57fcc00640 1 --2- 172.21.15.26:0/2848619399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57f80d1a50 0x7f57f815b560 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f57e800cb70 tx=0x7f57e800a430 comp rx=0 tx=0).stop 2024-04-26T21:10:30.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.497+0000 7f57f7e00640 1 -- 172.21.15.26:0/2848619399 reap_dead start 2024-04-26T21:10:30.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.498+0000 7f57fcc00640 1 -- 172.21.15.26:0/2848619399 shutdown_connections 2024-04-26T21:10:30.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.498+0000 7f57fcc00640 1 -- 172.21.15.26:0/2848619399 >> 172.21.15.26:0/2848619399 conn(0x7f57f80d5060 msgr2=0x7f57f806e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:30.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.498+0000 7f57fcc00640 1 -- 172.21.15.26:0/2848619399 shutdown_connections 2024-04-26T21:10:30.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.498+0000 7f57fcc00640 1 -- 172.21.15.26:0/2848619399 wait complete. 2024-04-26T21:10:30.517 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-26T21:10:30.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.719+0000 7fa3abe00640 1 Processor -- start 2024-04-26T21:10:30.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.719+0000 7fa3abe00640 1 -- start start 2024-04-26T21:10:30.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.720+0000 7fa3abe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3ac145c20 0x7fa3ac0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:30.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.720+0000 7fa3abe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa3ac0d1840 0x7fa3ac071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:30.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.720+0000 7fa3abe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3ac0cf690 con 0x7fa3ac146450 2024-04-26T21:10:30.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.720+0000 7fa3abe00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fa3ac0ced10 con 0x7fa3ac0d1840 2024-04-26T21:10:30.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.720+0000 7fa3abe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa3ac0cf930 con 0x7fa3ac145c20 2024-04-26T21:10:30.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.721+0000 7fa3ab400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa3ac0d1840 0x7fa3ac071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:30.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.721+0000 7fa3aaa00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3ac145c20 0x7fa3ac0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:30.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.721+0000 7fa3ab400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa3ac0d1840 0x7fa3ac071fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48412/0 (socket says 172.21.15.26:48412) 2024-04-26T21:10:30.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.721+0000 7fa3ab400640 1 -- 172.21.15.26:0/786705623 learned_addr learned my addr 172.21.15.26:0/786705623 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:30.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.722+0000 7fa3ab400640 1 -- 172.21.15.26:0/786705623 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3ac145c20 msgr2=0x7fa3ac0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:30.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.722+0000 7fa3ab400640 1 --2- 172.21.15.26:0/786705623 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3ac145c20 0x7fa3ac0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:30.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.722+0000 7fa3ab400640 1 -- 172.21.15.26:0/786705623 >> v1:172.21.15.86:6789/0 conn(0x7fa3ac146450 legacy=0x7fa3ac0d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:30.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.722+0000 7fa3ab400640 1 -- 172.21.15.26:0/786705623 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ac0cfc10 con 0x7fa3ac0d1840 2024-04-26T21:10:30.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.723+0000 7fa3ab400640 1 --2- 172.21.15.26:0/786705623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3ac0d1840 0x7fa3ac071fe0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009e50 tx=0x7fa3a402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc3eb3d68063c40 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:30.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.723+0000 7fa3a9600640 1 -- 172.21.15.26:0/786705623 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4036070 con 0x7fa3ac0d1840 2024-04-26T21:10:30.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.723+0000 7fa3a9600640 1 -- 172.21.15.26:0/786705623 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a4031600 con 0x7fa3ac0d1840 2024-04-26T21:10:30.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.723+0000 7fa3a9600640 1 -- 172.21.15.26:0/786705623 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a40319d0 con 0x7fa3ac0d1840 2024-04-26T21:10:30.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.724+0000 7fa3abe00640 1 -- 172.21.15.26:0/786705623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3ac0d1840 msgr2=0x7fa3ac071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.724+0000 7fa3abe00640 1 --2- 172.21.15.26:0/786705623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3ac0d1840 0x7fa3ac071fe0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009e50 tx=0x7fa3a402a710 comp rx=0 tx=0).stop 2024-04-26T21:10:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.724+0000 7fa3abe00640 1 -- 172.21.15.26:0/786705623 shutdown_connections 2024-04-26T21:10:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.724+0000 7fa3abe00640 1 --2- 172.21.15.26:0/786705623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3ac0d1840 0x7fa3ac071fe0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.724+0000 7fa3abe00640 1 --2- 172.21.15.26:0/786705623 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3ac145c20 0x7fa3ac0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.724+0000 7fa3abe00640 1 -- 172.21.15.26:0/786705623 >> 172.21.15.26:0/786705623 conn(0x7fa3ac06c690 msgr2=0x7fa3ac06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.724+0000 7fa3abe00640 1 -- 172.21.15.26:0/786705623 shutdown_connections 2024-04-26T21:10:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.724+0000 7fa3abe00640 1 -- 172.21.15.26:0/786705623 wait complete. 2024-04-26T21:10:30.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.725+0000 7fa3abe00640 1 Processor -- start 2024-04-26T21:10:30.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.726+0000 7fa3abe00640 1 -- start start 2024-04-26T21:10:30.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.727+0000 7fa3abe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3ac0d1840 0x7fa3ac15deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:30.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.727+0000 7fa3abe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3ac145c20 0x7fa3ac15d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:30.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.727+0000 7fa3abe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3ac146450 0x7fa3ac15bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:30.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.727+0000 7fa3abe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac146000 con 0x7fa3ac146450 2024-04-26T21:10:30.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.727+0000 7fa3abe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac0d01d0 con 0x7fa3ac0d1840 2024-04-26T21:10:30.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.727+0000 7fa3abe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa3ac15ce00 con 0x7fa3ac145c20 2024-04-26T21:10:30.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.727+0000 7fa3ab400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3ac146450 0x7fa3ac15bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:30.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.727+0000 7fa3ab400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3ac146450 0x7fa3ac15bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48414/0 (socket says 172.21.15.26:48414) 2024-04-26T21:10:30.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.727+0000 7fa3ab400640 1 -- 172.21.15.26:0/4164210121 learned_addr learned my addr 172.21.15.26:0/4164210121 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:30.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.727+0000 7fa3aa000640 1 --2- 172.21.15.26:0/4164210121 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3ac145c20 0x7fa3ac15d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:30.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.727+0000 7fa3aaa00640 1 --2- 172.21.15.26:0/4164210121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3ac0d1840 0x7fa3ac15deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:30.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.727+0000 7fa3ab400640 1 -- 172.21.15.26:0/4164210121 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3ac145c20 msgr2=0x7fa3ac15d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:30.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.727+0000 7fa3ab400640 1 --2- 172.21.15.26:0/4164210121 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3ac145c20 0x7fa3ac15d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:30.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.727+0000 7fa3ab400640 1 -- 172.21.15.26:0/4164210121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3ac0d1840 msgr2=0x7fa3ac15deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:30.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.727+0000 7fa3ab400640 1 --2- 172.21.15.26:0/4164210121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3ac0d1840 0x7fa3ac15deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:30.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.728+0000 7fa3ab400640 1 -- 172.21.15.26:0/4164210121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ac15caf0 con 0x7fa3ac146450 2024-04-26T21:10:30.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.728+0000 7fa3ab400640 1 --2- 172.21.15.26:0/4164210121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3ac146450 0x7fa3ac15bac0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4034040 tx=0x7fa3a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:30.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.728+0000 7fa3a2a00640 1 -- 172.21.15.26:0/4164210121 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4036050 con 0x7fa3ac146450 2024-04-26T21:10:30.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.728+0000 7fa3a2a00640 1 -- 172.21.15.26:0/4164210121 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a4042070 con 0x7fa3ac146450 2024-04-26T21:10:30.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.728+0000 7fa3a2a00640 1 -- 172.21.15.26:0/4164210121 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4009510 con 0x7fa3ac146450 2024-04-26T21:10:30.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.728+0000 7fa3abe00640 1 -- 172.21.15.26:0/4164210121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3ac15e540 con 0x7fa3ac146450 2024-04-26T21:10:30.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.728+0000 7fa3abe00640 1 -- 172.21.15.26:0/4164210121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3ac15d050 con 0x7fa3ac146450 2024-04-26T21:10:30.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.730+0000 7fa3abe00640 1 -- 172.21.15.26:0/4164210121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3ac06a700 con 0x7fa3ac146450 2024-04-26T21:10:30.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.731+0000 7fa3a2a00640 1 -- 172.21.15.26:0/4164210121 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3a4007bf0 con 0x7fa3ac146450 2024-04-26T21:10:30.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.731+0000 7fa3a2a00640 1 --2- 172.21.15.26:0/4164210121 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa38c06c8f0 0x7fa38c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:30.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.732+0000 7fa3a2a00640 1 -- 172.21.15.26:0/4164210121 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7fa3a40ac780 con 0x7fa3ac146450 2024-04-26T21:10:30.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.732+0000 7fa3aaa00640 1 --2- 172.21.15.26:0/4164210121 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa38c06c8f0 0x7fa38c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:30.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.733+0000 7fa3aaa00640 1 --2- 172.21.15.26:0/4164210121 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa38c06c8f0 0x7fa38c06edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009ad0 tx=0x7fa39c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:30.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.736+0000 7fa3a2a00640 1 -- 172.21.15.26:0/4164210121 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa3a4045360 con 0x7fa3ac146450 2024-04-26T21:10:30.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.942+0000 7fa3abe00640 1 -- 172.21.15.26:0/4164210121 --> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fa3ac15ea60 con 0x7fa38c06c8f0 2024-04-26T21:10:30.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.943+0000 7fa3a2a00640 1 -- 172.21.15.26:0/4164210121 <== mgr.4104 v2:172.21.15.26:6832/3130430358 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa39c0119a0 con 0x7fa38c06c8f0 2024-04-26T21:10:30.945 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 0 to deep-scrub 2024-04-26T21:10:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.945+0000 7fa3abe00640 1 -- 172.21.15.26:0/4164210121 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa38c06c8f0 msgr2=0x7fa38c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.945+0000 7fa3abe00640 1 --2- 172.21.15.26:0/4164210121 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa38c06c8f0 0x7fa38c06edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009ad0 tx=0x7fa39c008040 comp rx=0 tx=0).stop 2024-04-26T21:10:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.945+0000 7fa3abe00640 1 -- 172.21.15.26:0/4164210121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3ac146450 msgr2=0x7fa3ac15bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.945+0000 7fa3abe00640 1 --2- 172.21.15.26:0/4164210121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3ac146450 0x7fa3ac15bac0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4034040 tx=0x7fa3a4004a40 comp rx=0 tx=0).stop 2024-04-26T21:10:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.946+0000 7fa3abe00640 1 -- 172.21.15.26:0/4164210121 shutdown_connections 2024-04-26T21:10:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.946+0000 7fa3abe00640 1 --2- 172.21.15.26:0/4164210121 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa38c06c8f0 0x7fa38c06edb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.946+0000 7fa3abe00640 1 --2- 172.21.15.26:0/4164210121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3ac146450 0x7fa3ac15bac0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.946+0000 7fa3abe00640 1 --2- 172.21.15.26:0/4164210121 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3ac145c20 0x7fa3ac15d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.946+0000 7fa3abe00640 1 --2- 172.21.15.26:0/4164210121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3ac0d1840 0x7fa3ac15deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.946+0000 7fa3abe00640 1 -- 172.21.15.26:0/4164210121 >> 172.21.15.26:0/4164210121 conn(0x7fa3ac06c690 msgr2=0x7fa3ac06ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.946+0000 7fa3abe00640 1 -- 172.21.15.26:0/4164210121 shutdown_connections 2024-04-26T21:10:30.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:30.946+0000 7fa3abe00640 1 -- 172.21.15.26:0/4164210121 wait complete. 2024-04-26T21:10:30.967 INFO:tasks.ceph:Scrubbing osd.1 2024-04-26T21:10:30.967 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:10:31.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.187+0000 7f80a9e00640 1 Processor -- start 2024-04-26T21:10:31.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.188+0000 7f80a9e00640 1 -- start start 2024-04-26T21:10:31.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.189+0000 7f80a9e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f80a40b2d30 0x7f80a40b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.189+0000 7f80a9e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80a40b4ad0 0x7f80a40b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.189+0000 7f80a9e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80a40b4030 con 0x7f80a40aeb80 2024-04-26T21:10:31.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.189+0000 7f80a9e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f80a41608b0 con 0x7f80a40b2d30 2024-04-26T21:10:31.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.189+0000 7f80a9e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f80a40b3650 con 0x7f80a40b4ad0 2024-04-26T21:10:31.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.190+0000 7f80a2a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f80a40b2d30 0x7f80a40b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.190+0000 7f80a2a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f80a40b2d30 0x7f80a40b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48418/0 (socket says 172.21.15.26:48418) 2024-04-26T21:10:31.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.190+0000 7f80a2a00640 1 -- 172.21.15.26:0/2531614611 learned_addr learned my addr 172.21.15.26:0/2531614611 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:31.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.190+0000 7f80a3400640 1 --2- 172.21.15.26:0/2531614611 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80a40b4ad0 0x7f80a40b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.190+0000 7f80a2a00640 1 -- 172.21.15.26:0/2531614611 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80a40b4ad0 msgr2=0x7f80a40b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:31.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.190+0000 7f80a2a00640 1 --2- 172.21.15.26:0/2531614611 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80a40b4ad0 0x7f80a40b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.190+0000 7f80a2a00640 1 -- 172.21.15.26:0/2531614611 >> v1:172.21.15.86:6789/0 conn(0x7f80a40aeb80 legacy=0x7f80a415f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.191+0000 7f80a2a00640 1 -- 172.21.15.26:0/2531614611 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a41602f0 con 0x7f80a40b2d30 2024-04-26T21:10:31.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.192+0000 7f80a2a00640 1 --2- 172.21.15.26:0/2531614611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80a40b2d30 0x7f80a40b3110 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f80940098a0 tx=0x7f809402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f138463d8c328 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:31.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.192+0000 7f80a1600640 1 -- 172.21.15.26:0/2531614611 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8094037070 con 0x7f80a40b2d30 2024-04-26T21:10:31.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.192+0000 7f80a1600640 1 -- 172.21.15.26:0/2531614611 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8094032440 con 0x7f80a40b2d30 2024-04-26T21:10:31.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.192+0000 7f80a1600640 1 -- 172.21.15.26:0/2531614611 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8094032810 con 0x7f80a40b2d30 2024-04-26T21:10:31.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.192+0000 7f80a9e00640 1 -- 172.21.15.26:0/2531614611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80a40b2d30 msgr2=0x7f80a40b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.193+0000 7f80a9e00640 1 --2- 172.21.15.26:0/2531614611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80a40b2d30 0x7f80a40b3110 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f80940098a0 tx=0x7f809402a2a0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.193+0000 7f80a9e00640 1 -- 172.21.15.26:0/2531614611 shutdown_connections 2024-04-26T21:10:31.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.193+0000 7f80a9e00640 1 --2- 172.21.15.26:0/2531614611 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80a40b4ad0 0x7f80a40b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.193+0000 7f80a9e00640 1 --2- 172.21.15.26:0/2531614611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80a40b2d30 0x7f80a40b3110 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.193+0000 7f80a9e00640 1 -- 172.21.15.26:0/2531614611 >> 172.21.15.26:0/2531614611 conn(0x7f80a406c420 msgr2=0x7f80a406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:31.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.193+0000 7f80a9e00640 1 -- 172.21.15.26:0/2531614611 shutdown_connections 2024-04-26T21:10:31.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.193+0000 7f80a9e00640 1 -- 172.21.15.26:0/2531614611 wait complete. 2024-04-26T21:10:31.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.195+0000 7f80a9e00640 1 Processor -- start 2024-04-26T21:10:31.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.195+0000 7f80a9e00640 1 -- start start 2024-04-26T21:10:31.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.196+0000 7f80a9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80a40aeb80 0x7f80a41e8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.196+0000 7f80a9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a40b2d30 0x7f80a40cf420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.196+0000 7f80a9e00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80a40b4ad0 0x7f80a415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.196+0000 7f80a9e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f80a40b45b0 con 0x7f80a40aeb80 2024-04-26T21:10:31.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.196+0000 7f80a9e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80a40b42d0 con 0x7f80a40b2d30 2024-04-26T21:10:31.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.196+0000 7f80a9e00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f80a415ff70 con 0x7f80a40b4ad0 2024-04-26T21:10:31.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.196+0000 7f80a2a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80a40aeb80 0x7f80a41e8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.196+0000 7f80a2a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80a40aeb80 0x7f80a41e8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48426/0 (socket says 172.21.15.26:48426) 2024-04-26T21:10:31.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.196+0000 7f80a2a00640 1 -- 172.21.15.26:0/2758822383 learned_addr learned my addr 172.21.15.26:0/2758822383 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:31.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.196+0000 7f80a2000640 1 --2- 172.21.15.26:0/2758822383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a40b2d30 0x7f80a40cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.196+0000 7f80a2a00640 1 -- 172.21.15.26:0/2758822383 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80a40b4ad0 msgr2=0x7f80a415e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.196+0000 7f80a2a00640 1 --2- 172.21.15.26:0/2758822383 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80a40b4ad0 0x7f80a415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.196+0000 7f80a2a00640 1 -- 172.21.15.26:0/2758822383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a40b2d30 msgr2=0x7f80a40cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.196+0000 7f80a2a00640 1 --2- 172.21.15.26:0/2758822383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a40b2d30 0x7f80a40cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.196+0000 7f80a2a00640 1 -- 172.21.15.26:0/2758822383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a40cfc70 con 0x7f80a40aeb80 2024-04-26T21:10:31.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.196+0000 7f80a2000640 1 --2- 172.21.15.26:0/2758822383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a40b2d30 0x7f80a40cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:31.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.197+0000 7f80a2a00640 1 --2- 172.21.15.26:0/2758822383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80a40aeb80 0x7f80a41e8ce0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8094000c00 tx=0x7f80940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:31.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.197+0000 7f809b400640 1 -- 172.21.15.26:0/2758822383 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8094037040 con 0x7f80a40aeb80 2024-04-26T21:10:31.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.197+0000 7f809b400640 1 -- 172.21.15.26:0/2758822383 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80940304c0 con 0x7f80a40aeb80 2024-04-26T21:10:31.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.197+0000 7f809b400640 1 -- 172.21.15.26:0/2758822383 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80940308b0 con 0x7f80a40aeb80 2024-04-26T21:10:31.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.197+0000 7f80a9e00640 1 -- 172.21.15.26:0/2758822383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a40cff50 con 0x7f80a40aeb80 2024-04-26T21:10:31.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.197+0000 7f80a9e00640 1 -- 172.21.15.26:0/2758822383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a40d0510 con 0x7f80a40aeb80 2024-04-26T21:10:31.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.198+0000 7f80a9e00640 1 -- 172.21.15.26:0/2758822383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f80a406a700 con 0x7f80a40aeb80 2024-04-26T21:10:31.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.199+0000 7f809b400640 1 -- 172.21.15.26:0/2758822383 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8094044020 con 0x7f80a40aeb80 2024-04-26T21:10:31.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.200+0000 7f809b400640 1 --2- 172.21.15.26:0/2758822383 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f808406c8f0 0x7f808406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.200+0000 7f809b400640 1 -- 172.21.15.26:0/2758822383 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f80940ab870 con 0x7f80a40aeb80 2024-04-26T21:10:31.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.201+0000 7f80a2000640 1 --2- 172.21.15.26:0/2758822383 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f808406c8f0 0x7f808406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.201+0000 7f809b400640 1 --2- 172.21.15.26:0/2758822383 >> [v2:172.21.15.26:6800/398686833,v1:172.21.15.26:6801/398686833] conn(0x7f80840727b0 0x7f8084074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.201+0000 7f809b400640 1 -- 172.21.15.26:0/2758822383 --> [v2:172.21.15.26:6800/398686833,v1:172.21.15.26:6801/398686833] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8094044e80 con 0x7f80840727b0 2024-04-26T21:10:31.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.201+0000 7f809b400640 1 -- 172.21.15.26:0/2758822383 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f80940b02a0 con 0x7f80a40aeb80 2024-04-26T21:10:31.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.201+0000 7f80a3400640 1 --2- 172.21.15.26:0/2758822383 >> [v2:172.21.15.26:6800/398686833,v1:172.21.15.26:6801/398686833] conn(0x7f80840727b0 0x7f8084074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.201+0000 7f80a2000640 1 --2- 172.21.15.26:0/2758822383 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f808406c8f0 0x7f808406edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f80a40b2670 tx=0x7f8090009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:31.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.202+0000 7f80a3400640 1 --2- 172.21.15.26:0/2758822383 >> [v2:172.21.15.26:6800/398686833,v1:172.21.15.26:6801/398686833] conn(0x7f80840727b0 0x7f8084074bd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:31.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.202+0000 7f809b400640 1 -- 172.21.15.26:0/2758822383 <== osd.1 v2:172.21.15.26:6800/398686833 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8094044e80 con 0x7f80840727b0 2024-04-26T21:10:31.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.217+0000 7f80a9e00640 1 -- 172.21.15.26:0/2758822383 --> [v2:172.21.15.26:6800/398686833,v1:172.21.15.26:6801/398686833] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f80a40d0750 con 0x7f80840727b0 2024-04-26T21:10:31.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.218+0000 7f809b400640 1 -- 172.21.15.26:0/2758822383 <== osd.1 v2:172.21.15.26:6800/398686833 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f80a40d0750 con 0x7f80840727b0 2024-04-26T21:10:31.220 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-26T21:10:31.220 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:10:31.220 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-26T21:10:31.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.218+0000 7f80a9e00640 1 -- 172.21.15.26:0/2758822383 >> [v2:172.21.15.26:6800/398686833,v1:172.21.15.26:6801/398686833] conn(0x7f80840727b0 msgr2=0x7f8084074bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.218+0000 7f80a9e00640 1 --2- 172.21.15.26:0/2758822383 >> [v2:172.21.15.26:6800/398686833,v1:172.21.15.26:6801/398686833] conn(0x7f80840727b0 0x7f8084074bd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.219+0000 7f80a9e00640 1 -- 172.21.15.26:0/2758822383 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f808406c8f0 msgr2=0x7f808406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.219+0000 7f80a9e00640 1 --2- 172.21.15.26:0/2758822383 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f808406c8f0 0x7f808406edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f80a40b2670 tx=0x7f8090009040 comp rx=0 tx=0).stop 2024-04-26T21:10:31.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.219+0000 7f80a9e00640 1 -- 172.21.15.26:0/2758822383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80a40aeb80 msgr2=0x7f80a41e8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.219+0000 7f80a9e00640 1 --2- 172.21.15.26:0/2758822383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80a40aeb80 0x7f80a41e8ce0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8094000c00 tx=0x7f80940046c0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.219+0000 7f80a3400640 1 -- 172.21.15.26:0/2758822383 reap_dead start 2024-04-26T21:10:31.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.219+0000 7f80a9e00640 1 -- 172.21.15.26:0/2758822383 shutdown_connections 2024-04-26T21:10:31.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.219+0000 7f80a9e00640 1 -- 172.21.15.26:0/2758822383 >> 172.21.15.26:0/2758822383 conn(0x7f80a406c420 msgr2=0x7f80a4160a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:31.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.219+0000 7f80a9e00640 1 -- 172.21.15.26:0/2758822383 shutdown_connections 2024-04-26T21:10:31.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.219+0000 7f80a9e00640 1 -- 172.21.15.26:0/2758822383 wait complete. 2024-04-26T21:10:31.239 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-26T21:10:31.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.446+0000 7f47f3000640 1 Processor -- start 2024-04-26T21:10:31.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.446+0000 7f47f3000640 1 -- start start 2024-04-26T21:10:31.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.448+0000 7f47f3000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec0d1c80 0x7f47ec0d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.448+0000 7f47f3000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f47ec162ba0 0x7f47ec0d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.448+0000 7f47f3000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47ec0b61c0 con 0x7f47ec0d2060 2024-04-26T21:10:31.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.448+0000 7f47f3000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f47ec0b5f70 con 0x7f47ec162ba0 2024-04-26T21:10:31.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.448+0000 7f47f3000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f47ec0d3830 con 0x7f47ec0d1c80 2024-04-26T21:10:31.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.449+0000 7f47f1c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec0d1c80 0x7f47ec0d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.449+0000 7f47f2600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f47ec162ba0 0x7f47ec0d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.449+0000 7f47f1c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec0d1c80 0x7f47ec0d3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.26:51916/0 (socket says 172.21.15.26:51916) 2024-04-26T21:10:31.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.449+0000 7f47f2600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f47ec162ba0 0x7f47ec0d0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.26:43326/0 (socket says 172.21.15.26:43326) 2024-04-26T21:10:31.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.449+0000 7f47f1c00640 1 -- 172.21.15.26:0/3202360001 learned_addr learned my addr 172.21.15.26:0/3202360001 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:31.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.449+0000 7f47f1c00640 1 -- 172.21.15.26:0/3202360001 >> v2:172.21.15.86:3300/0 conn(0x7f47ec162ba0 msgr2=0x7f47ec0d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:31.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.449+0000 7f47f1c00640 1 --2- 172.21.15.26:0/3202360001 >> v2:172.21.15.86:3300/0 conn(0x7f47ec162ba0 0x7f47ec0d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.449+0000 7f47f1c00640 1 -- 172.21.15.26:0/3202360001 >> v1:172.21.15.86:6789/0 conn(0x7f47ec0d2060 legacy=0x7f47ec162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.450+0000 7f47f1c00640 1 -- 172.21.15.26:0/3202360001 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ec0b64a0 con 0x7f47ec0d1c80 2024-04-26T21:10:31.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.451+0000 7f47f1c00640 1 --2- 172.21.15.26:0/3202360001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec0d1c80 0x7f47ec0d3cc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009870 tx=0x7f47dc02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8efcb2e6a4cf5f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:31.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.452+0000 7f47ebe00640 1 -- 172.21.15.26:0/3202360001 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc037070 con 0x7f47ec0d1c80 2024-04-26T21:10:31.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.452+0000 7f47ebe00640 1 -- 172.21.15.26:0/3202360001 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47dc032440 con 0x7f47ec0d1c80 2024-04-26T21:10:31.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.452+0000 7f47ebe00640 1 -- 172.21.15.26:0/3202360001 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc032810 con 0x7f47ec0d1c80 2024-04-26T21:10:31.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.453+0000 7f47f3000640 1 -- 172.21.15.26:0/3202360001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec0d1c80 msgr2=0x7f47ec0d3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.453+0000 7f47f3000640 1 --2- 172.21.15.26:0/3202360001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec0d1c80 0x7f47ec0d3cc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009870 tx=0x7f47dc02a1c0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.453+0000 7f47f3000640 1 -- 172.21.15.26:0/3202360001 shutdown_connections 2024-04-26T21:10:31.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.453+0000 7f47f3000640 1 --2- 172.21.15.26:0/3202360001 >> v2:172.21.15.86:3300/0 conn(0x7f47ec162ba0 0x7f47ec0d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.453+0000 7f47f3000640 1 --2- 172.21.15.26:0/3202360001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec0d1c80 0x7f47ec0d3cc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.453+0000 7f47f3000640 1 -- 172.21.15.26:0/3202360001 >> 172.21.15.26:0/3202360001 conn(0x7f47ec0d5020 msgr2=0x7f47ec0d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:31.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.453+0000 7f47f3000640 1 -- 172.21.15.26:0/3202360001 shutdown_connections 2024-04-26T21:10:31.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.453+0000 7f47f3000640 1 -- 172.21.15.26:0/3202360001 wait complete. 2024-04-26T21:10:31.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.455+0000 7f47f3000640 1 Processor -- start 2024-04-26T21:10:31.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.455+0000 7f47f3000640 1 -- start start 2024-04-26T21:10:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.456+0000 7f47f3000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ec0d1c80 0x7f47ec153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.456+0000 7f47f3000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec0d2060 0x7f47ec153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.456+0000 7f47f3000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec162ba0 0x7f47ec0b96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.456+0000 7f47f3000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec0d5cc0 con 0x7f47ec0d1c80 2024-04-26T21:10:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.456+0000 7f47f3000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec0d5b40 con 0x7f47ec162ba0 2024-04-26T21:10:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.456+0000 7f47f3000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f47ec0c8b70 con 0x7f47ec0d2060 2024-04-26T21:10:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.456+0000 7f47f1c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ec0d1c80 0x7f47ec153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.456+0000 7f47f1c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ec0d1c80 0x7f47ec153050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48434/0 (socket says 172.21.15.26:48434) 2024-04-26T21:10:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.456+0000 7f47f1c00640 1 -- 172.21.15.26:0/537744451 learned_addr learned my addr 172.21.15.26:0/537744451 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:31.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.457+0000 7f47f1200640 1 --2- 172.21.15.26:0/537744451 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec0d2060 0x7f47ec153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.457+0000 7f47f1c00640 1 -- 172.21.15.26:0/537744451 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec0d2060 msgr2=0x7f47ec153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.457+0000 7f47f1c00640 1 --2- 172.21.15.26:0/537744451 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec0d2060 0x7f47ec153630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.457+0000 7f47f1c00640 1 -- 172.21.15.26:0/537744451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec162ba0 msgr2=0x7f47ec0b96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.457+0000 7f47f1c00640 1 --2- 172.21.15.26:0/537744451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec162ba0 0x7f47ec0b96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.457+0000 7f47f1c00640 1 -- 172.21.15.26:0/537744451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ec1f8810 con 0x7f47ec0d1c80 2024-04-26T21:10:31.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.457+0000 7f47f1c00640 1 --2- 172.21.15.26:0/537744451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ec0d1c80 0x7f47ec153050 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f47dc000c00 tx=0x7f47dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:31.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.457+0000 7f47e9600640 1 -- 172.21.15.26:0/537744451 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc037040 con 0x7f47ec0d1c80 2024-04-26T21:10:31.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.458+0000 7f47e9600640 1 -- 172.21.15.26:0/537744451 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47dc042070 con 0x7f47ec0d1c80 2024-04-26T21:10:31.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.458+0000 7f47e9600640 1 -- 172.21.15.26:0/537744451 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc030940 con 0x7f47ec0d1c80 2024-04-26T21:10:31.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.458+0000 7f47f3000640 1 -- 172.21.15.26:0/537744451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47ec1f8af0 con 0x7f47ec0d1c80 2024-04-26T21:10:31.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.458+0000 7f47f3000640 1 -- 172.21.15.26:0/537744451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47ec1f90b0 con 0x7f47ec0d1c80 2024-04-26T21:10:31.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.459+0000 7f47f3000640 1 -- 172.21.15.26:0/537744451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47ec06a700 con 0x7f47ec0d1c80 2024-04-26T21:10:31.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.460+0000 7f47e9600640 1 -- 172.21.15.26:0/537744451 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47dc02ad10 con 0x7f47ec0d1c80 2024-04-26T21:10:31.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.462+0000 7f47e9600640 1 --2- 172.21.15.26:0/537744451 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f47d006c8f0 0x7f47d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.462+0000 7f47e9600640 1 -- 172.21.15.26:0/537744451 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f47dc0abe20 con 0x7f47ec0d1c80 2024-04-26T21:10:31.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.462+0000 7f47f1200640 1 --2- 172.21.15.26:0/537744451 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f47d006c8f0 0x7f47d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.463+0000 7f47f1200640 1 --2- 172.21.15.26:0/537744451 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f47d006c8f0 0x7f47d006edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f47e0007fd0 tx=0x7f47e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:31.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.466+0000 7f47e9600640 1 -- 172.21.15.26:0/537744451 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f47dc07a540 con 0x7f47ec0d1c80 2024-04-26T21:10:31.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.669+0000 7f47f3000640 1 -- 172.21.15.26:0/537744451 --> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f47ec0cf120 con 0x7f47d006c8f0 2024-04-26T21:10:31.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.670+0000 7f47e9600640 1 -- 172.21.15.26:0/537744451 <== mgr.4104 v2:172.21.15.26:6832/3130430358 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f47ec0cf120 con 0x7f47d006c8f0 2024-04-26T21:10:31.671 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 1 to deep-scrub 2024-04-26T21:10:31.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.672+0000 7f47f3000640 1 -- 172.21.15.26:0/537744451 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f47d006c8f0 msgr2=0x7f47d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.672+0000 7f47f3000640 1 --2- 172.21.15.26:0/537744451 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f47d006c8f0 0x7f47d006edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f47e0007fd0 tx=0x7f47e0009040 comp rx=0 tx=0).stop 2024-04-26T21:10:31.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.672+0000 7f47f3000640 1 -- 172.21.15.26:0/537744451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ec0d1c80 msgr2=0x7f47ec153050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.672+0000 7f47f3000640 1 --2- 172.21.15.26:0/537744451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ec0d1c80 0x7f47ec153050 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f47dc000c00 tx=0x7f47dc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.673+0000 7f47f3000640 1 -- 172.21.15.26:0/537744451 shutdown_connections 2024-04-26T21:10:31.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.673+0000 7f47f3000640 1 --2- 172.21.15.26:0/537744451 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f47d006c8f0 0x7f47d006edb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.673+0000 7f47f3000640 1 --2- 172.21.15.26:0/537744451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec162ba0 0x7f47ec0b96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.673+0000 7f47f3000640 1 --2- 172.21.15.26:0/537744451 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec0d2060 0x7f47ec153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.673+0000 7f47f3000640 1 --2- 172.21.15.26:0/537744451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47ec0d1c80 0x7f47ec153050 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.673+0000 7f47f3000640 1 -- 172.21.15.26:0/537744451 >> 172.21.15.26:0/537744451 conn(0x7f47ec0d5020 msgr2=0x7f47ec0d5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:31.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.673+0000 7f47f3000640 1 -- 172.21.15.26:0/537744451 shutdown_connections 2024-04-26T21:10:31.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.673+0000 7f47f3000640 1 -- 172.21.15.26:0/537744451 wait complete. 2024-04-26T21:10:31.694 INFO:tasks.ceph:Scrubbing osd.2 2024-04-26T21:10:31.694 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:10:31.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.909+0000 7f0721000640 1 Processor -- start 2024-04-26T21:10:31.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.909+0000 7f0721000640 1 -- start start 2024-04-26T21:10:31.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.910+0000 7f0721000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f071c162940 0x7f071c0b8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.910+0000 7f0721000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f071c0d1a80 0x7f071c0d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.910+0000 7f0721000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f071c0b6350 con 0x7f071c0cfdc0 2024-04-26T21:10:31.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.910+0000 7f0721000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f071c0d2430 con 0x7f071c162940 2024-04-26T21:10:31.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.910+0000 7f0721000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f071c0d2d80 con 0x7f071c0d1a80 2024-04-26T21:10:31.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.911+0000 7f071a800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f071c0cfdc0 0x7f071c0d01f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33436/0 (socket says 172.21.15.26:33436) 2024-04-26T21:10:31.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.911+0000 7f071a800640 1 -- 172.21.15.26:0/718114768 learned_addr learned my addr 172.21.15.26:0/718114768 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:31.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.911+0000 7f0719e00640 1 --2- 172.21.15.26:0/718114768 >> v2:172.21.15.26:3300/0 conn(0x7f071c162940 0x7f071c0b8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.911+0000 7f0719400640 1 --2- 172.21.15.26:0/718114768 >> v2:172.21.15.86:3300/0 conn(0x7f071c0d1a80 0x7f071c0d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.911+0000 7f0718a00640 1 -- 172.21.15.26:0/718114768 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2927440081 0 0) 0x7f0714001110 con 0x7f071c0cfdc0 2024-04-26T21:10:31.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.911+0000 7f0718a00640 1 -- 172.21.15.26:0/718114768 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0700003520 con 0x7f071c0cfdc0 2024-04-26T21:10:31.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.912+0000 7f0719e00640 1 -- 172.21.15.26:0/718114768 >> v2:172.21.15.86:3300/0 conn(0x7f071c0d1a80 msgr2=0x7f071c0d1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:31.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.912+0000 7f0719e00640 1 --2- 172.21.15.26:0/718114768 >> v2:172.21.15.86:3300/0 conn(0x7f071c0d1a80 0x7f071c0d1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.912+0000 7f0719e00640 1 -- 172.21.15.26:0/718114768 >> v1:172.21.15.26:6789/0 conn(0x7f071c0cfdc0 legacy=0x7f071c0d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.912+0000 7f0719e00640 1 -- 172.21.15.26:0/718114768 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071c0d0a80 con 0x7f071c162940 2024-04-26T21:10:31.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.913+0000 7f0719e00640 1 --2- 172.21.15.26:0/718114768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f071c162940 0x7f071c0b8000 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f071c074730 tx=0x7f070c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9328b7e05fe1782 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:31.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.913+0000 7f0718a00640 1 -- 172.21.15.26:0/718114768 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c007740 con 0x7f071c162940 2024-04-26T21:10:31.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.913+0000 7f0718a00640 1 -- 172.21.15.26:0/718114768 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f070c0078e0 con 0x7f071c162940 2024-04-26T21:10:31.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.913+0000 7f0718a00640 1 -- 172.21.15.26:0/718114768 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c007cb0 con 0x7f071c162940 2024-04-26T21:10:31.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.914+0000 7f0721000640 1 -- 172.21.15.26:0/718114768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f071c162940 msgr2=0x7f071c0b8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.914+0000 7f0721000640 1 --2- 172.21.15.26:0/718114768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f071c162940 0x7f071c0b8000 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f071c074730 tx=0x7f070c02a680 comp rx=0 tx=0).stop 2024-04-26T21:10:31.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.914+0000 7f0721000640 1 -- 172.21.15.26:0/718114768 shutdown_connections 2024-04-26T21:10:31.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.914+0000 7f0721000640 1 --2- 172.21.15.26:0/718114768 >> v2:172.21.15.86:3300/0 conn(0x7f071c0d1a80 0x7f071c0d1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.914+0000 7f0721000640 1 --2- 172.21.15.26:0/718114768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f071c162940 0x7f071c0b8000 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.914+0000 7f0721000640 1 -- 172.21.15.26:0/718114768 >> 172.21.15.26:0/718114768 conn(0x7f071c09e490 msgr2=0x7f071c0d51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:31.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.914+0000 7f0721000640 1 -- 172.21.15.26:0/718114768 shutdown_connections 2024-04-26T21:10:31.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.914+0000 7f0721000640 1 -- 172.21.15.26:0/718114768 wait complete. 2024-04-26T21:10:31.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.916+0000 7f0721000640 1 Processor -- start 2024-04-26T21:10:31.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.916+0000 7f0721000640 1 -- start start 2024-04-26T21:10:31.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.917+0000 7f0721000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f071c0cfdc0 0x7f071c159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.917+0000 7f0721000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f071c0d1a80 0x7f071c15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.917+0000 7f0721000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f071c162940 0x7f071c15c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.917+0000 7f0721000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f071c0d2fd0 con 0x7f071c0cfdc0 2024-04-26T21:10:31.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.917+0000 7f0721000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f071c0d1040 con 0x7f071c162940 2024-04-26T21:10:31.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.917+0000 7f0721000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f071c162d20 con 0x7f071c0d1a80 2024-04-26T21:10:31.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.917+0000 7f0719e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f071c0cfdc0 0x7f071c159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.917+0000 7f0719e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f071c0cfdc0 0x7f071c159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48452/0 (socket says 172.21.15.26:48452) 2024-04-26T21:10:31.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.917+0000 7f0719e00640 1 -- 172.21.15.26:0/4014988958 learned_addr learned my addr 172.21.15.26:0/4014988958 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:31.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.918+0000 7f071a800640 1 --2- 172.21.15.26:0/4014988958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f071c162940 0x7f071c15c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.918+0000 7f0719400640 1 --2- 172.21.15.26:0/4014988958 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f071c0d1a80 0x7f071c15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.918+0000 7f0719e00640 1 -- 172.21.15.26:0/4014988958 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f071c0d1a80 msgr2=0x7f071c15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.918+0000 7f0719e00640 1 --2- 172.21.15.26:0/4014988958 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f071c0d1a80 0x7f071c15c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.918+0000 7f0719e00640 1 -- 172.21.15.26:0/4014988958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f071c162940 msgr2=0x7f071c15c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.918+0000 7f0719e00640 1 --2- 172.21.15.26:0/4014988958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f071c162940 0x7f071c15c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.918+0000 7f0719e00640 1 -- 172.21.15.26:0/4014988958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071c0ca940 con 0x7f071c0cfdc0 2024-04-26T21:10:31.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.918+0000 7f0719400640 1 --2- 172.21.15.26:0/4014988958 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f071c0d1a80 0x7f071c15c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:31.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.918+0000 7f0719e00640 1 --2- 172.21.15.26:0/4014988958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f071c0cfdc0 0x7f071c159690 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f070c009e80 tx=0x7f070c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:31.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.919+0000 7f0712000640 1 -- 172.21.15.26:0/4014988958 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c03f070 con 0x7f071c0cfdc0 2024-04-26T21:10:31.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.919+0000 7f0712000640 1 -- 172.21.15.26:0/4014988958 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f070c031c40 con 0x7f071c0cfdc0 2024-04-26T21:10:31.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.919+0000 7f0721000640 1 -- 172.21.15.26:0/4014988958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f071c0cac20 con 0x7f071c0cfdc0 2024-04-26T21:10:31.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.919+0000 7f0712000640 1 -- 172.21.15.26:0/4014988958 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c009450 con 0x7f071c0cfdc0 2024-04-26T21:10:31.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.919+0000 7f0721000640 1 -- 172.21.15.26:0/4014988958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f071c0cb1e0 con 0x7f071c0cfdc0 2024-04-26T21:10:31.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.920+0000 7f0721000640 1 -- 172.21.15.26:0/4014988958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f071c06a700 con 0x7f071c0cfdc0 2024-04-26T21:10:31.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.921+0000 7f0712000640 1 -- 172.21.15.26:0/4014988958 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f070c043410 con 0x7f071c0cfdc0 2024-04-26T21:10:31.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.922+0000 7f0712000640 1 --2- 172.21.15.26:0/4014988958 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f06f806c8f0 0x7f06f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.922+0000 7f0712000640 1 -- 172.21.15.26:0/4014988958 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f070c0ad130 con 0x7f071c0cfdc0 2024-04-26T21:10:31.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.922+0000 7f0712000640 1 --2- 172.21.15.26:0/4014988958 >> [v2:172.21.15.26:6824/2316564464,v1:172.21.15.26:6825/2316564464] conn(0x7f06f80727b0 0x7f06f8074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:31.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.922+0000 7f0719400640 1 --2- 172.21.15.26:0/4014988958 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f06f806c8f0 0x7f06f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.922+0000 7f0712000640 1 -- 172.21.15.26:0/4014988958 --> [v2:172.21.15.26:6824/2316564464,v1:172.21.15.26:6825/2316564464] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f06f8075280 con 0x7f06f80727b0 2024-04-26T21:10:31.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.922+0000 7f0712000640 1 -- 172.21.15.26:0/4014988958 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f070c0ad4e0 con 0x7f071c0cfdc0 2024-04-26T21:10:31.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.923+0000 7f071a800640 1 --2- 172.21.15.26:0/4014988958 >> [v2:172.21.15.26:6824/2316564464,v1:172.21.15.26:6825/2316564464] conn(0x7f06f80727b0 0x7f06f8074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:31.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.923+0000 7f0719400640 1 --2- 172.21.15.26:0/4014988958 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f06f806c8f0 0x7f06f806edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f071c161970 tx=0x7f0704008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:31.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.923+0000 7f071a800640 1 --2- 172.21.15.26:0/4014988958 >> [v2:172.21.15.26:6824/2316564464,v1:172.21.15.26:6825/2316564464] conn(0x7f06f80727b0 0x7f06f8074bd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:31.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.924+0000 7f0712000640 1 -- 172.21.15.26:0/4014988958 <== osd.2 v2:172.21.15.26:6824/2316564464 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f06f8075280 con 0x7f06f80727b0 2024-04-26T21:10:31.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.939+0000 7f0721000640 1 -- 172.21.15.26:0/4014988958 --> [v2:172.21.15.26:6824/2316564464,v1:172.21.15.26:6825/2316564464] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f071c0cd7c0 con 0x7f06f80727b0 2024-04-26T21:10:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.939+0000 7f0712000640 1 -- 172.21.15.26:0/4014988958 <== osd.2 v2:172.21.15.26:6824/2316564464 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0714001810 con 0x7f06f80727b0 2024-04-26T21:10:31.941 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-26T21:10:31.941 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:10:31.941 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-26T21:10:31.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.940+0000 7f0721000640 1 -- 172.21.15.26:0/4014988958 >> [v2:172.21.15.26:6824/2316564464,v1:172.21.15.26:6825/2316564464] conn(0x7f06f80727b0 msgr2=0x7f06f8074bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.940+0000 7f0721000640 1 --2- 172.21.15.26:0/4014988958 >> [v2:172.21.15.26:6824/2316564464,v1:172.21.15.26:6825/2316564464] conn(0x7f06f80727b0 0x7f06f8074bd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:31.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.940+0000 7f0721000640 1 -- 172.21.15.26:0/4014988958 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f06f806c8f0 msgr2=0x7f06f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.940+0000 7f0721000640 1 --2- 172.21.15.26:0/4014988958 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f06f806c8f0 0x7f06f806edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f071c161970 tx=0x7f0704008040 comp rx=0 tx=0).stop 2024-04-26T21:10:31.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.940+0000 7f0721000640 1 -- 172.21.15.26:0/4014988958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f071c0cfdc0 msgr2=0x7f071c159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:31.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.940+0000 7f0721000640 1 --2- 172.21.15.26:0/4014988958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f071c0cfdc0 0x7f071c159690 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f070c009e80 tx=0x7f070c004a40 comp rx=0 tx=0).stop 2024-04-26T21:10:31.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.940+0000 7f071a800640 1 -- 172.21.15.26:0/4014988958 reap_dead start 2024-04-26T21:10:31.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.940+0000 7f0721000640 1 -- 172.21.15.26:0/4014988958 shutdown_connections 2024-04-26T21:10:31.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.940+0000 7f0721000640 1 -- 172.21.15.26:0/4014988958 >> 172.21.15.26:0/4014988958 conn(0x7f071c09e490 msgr2=0x7f071c06ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:31.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.940+0000 7f0721000640 1 -- 172.21.15.26:0/4014988958 shutdown_connections 2024-04-26T21:10:31.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:31.940+0000 7f0721000640 1 -- 172.21.15.26:0/4014988958 wait complete. 2024-04-26T21:10:31.962 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-26T21:10:32.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.176+0000 7f3a3d200640 1 Processor -- start 2024-04-26T21:10:32.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.176+0000 7f3a3d200640 1 -- start start 2024-04-26T21:10:32.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.177+0000 7f3a3d200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a380d1c80 0x7f3a380d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.177+0000 7f3a3d200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3a380d2060 0x7f3a38162490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.177+0000 7f3a3d200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a38162e60 con 0x7f3a381629d0 2024-04-26T21:10:32.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.177+0000 7f3a3d200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f3a380b5f70 con 0x7f3a380d2060 2024-04-26T21:10:32.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.177+0000 7f3a3d200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3a380d3830 con 0x7f3a380d1c80 2024-04-26T21:10:32.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.177+0000 7f3a36a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3a380d2060 0x7f3a38162490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.177+0000 7f3a36a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3a380d2060 0x7f3a38162490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48466/0 (socket says 172.21.15.26:48466) 2024-04-26T21:10:32.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.177+0000 7f3a36a00640 1 -- 172.21.15.26:0/615380600 learned_addr learned my addr 172.21.15.26:0/615380600 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:32.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.178+0000 7f3a37400640 1 --2- 172.21.15.26:0/615380600 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a380d1c80 0x7f3a380d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.178+0000 7f3a36a00640 1 -- 172.21.15.26:0/615380600 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a380d1c80 msgr2=0x7f3a380d3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:32.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.178+0000 7f3a36a00640 1 --2- 172.21.15.26:0/615380600 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a380d1c80 0x7f3a380d3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.178+0000 7f3a36a00640 1 -- 172.21.15.26:0/615380600 >> v1:172.21.15.86:6789/0 conn(0x7f3a381629d0 legacy=0x7f3a380c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:32.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.178+0000 7f3a36a00640 1 -- 172.21.15.26:0/615380600 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a380b61c0 con 0x7f3a380d2060 2024-04-26T21:10:32.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.179+0000 7f3a36a00640 1 --2- 172.21.15.26:0/615380600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a380d2060 0x7f3a38162490 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009870 tx=0x7f3a2402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958d503a8ae9855b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:32.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.179+0000 7f3a36000640 1 -- 172.21.15.26:0/615380600 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a24037070 con 0x7f3a380d2060 2024-04-26T21:10:32.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.179+0000 7f3a36000640 1 -- 172.21.15.26:0/615380600 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a24032440 con 0x7f3a380d2060 2024-04-26T21:10:32.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.179+0000 7f3a36000640 1 -- 172.21.15.26:0/615380600 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a24032810 con 0x7f3a380d2060 2024-04-26T21:10:32.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.180+0000 7f3a3d200640 1 -- 172.21.15.26:0/615380600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a380d2060 msgr2=0x7f3a38162490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.180+0000 7f3a3d200640 1 --2- 172.21.15.26:0/615380600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a380d2060 0x7f3a38162490 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009870 tx=0x7f3a2402a180 comp rx=0 tx=0).stop 2024-04-26T21:10:32.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.180+0000 7f3a3d200640 1 -- 172.21.15.26:0/615380600 shutdown_connections 2024-04-26T21:10:32.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.180+0000 7f3a3d200640 1 --2- 172.21.15.26:0/615380600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a380d2060 0x7f3a38162490 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.180+0000 7f3a3d200640 1 --2- 172.21.15.26:0/615380600 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a380d1c80 0x7f3a380d3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.180+0000 7f3a3d200640 1 -- 172.21.15.26:0/615380600 >> 172.21.15.26:0/615380600 conn(0x7f3a380d5020 msgr2=0x7f3a380d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:32.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.180+0000 7f3a3d200640 1 -- 172.21.15.26:0/615380600 shutdown_connections 2024-04-26T21:10:32.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.180+0000 7f3a3d200640 1 -- 172.21.15.26:0/615380600 wait complete. 2024-04-26T21:10:32.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.182+0000 7f3a3d200640 1 Processor -- start 2024-04-26T21:10:32.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.182+0000 7f3a3d200640 1 -- start start 2024-04-26T21:10:32.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.183+0000 7f3a3d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a380d1c80 0x7f3a38154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.183+0000 7f3a3d200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a380d2060 0x7f3a3815fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.183+0000 7f3a3d200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a381629d0 0x7f3a38160110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.183+0000 7f3a3d200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3a38074790 con 0x7f3a381629d0 2024-04-26T21:10:32.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.183+0000 7f3a3d200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a38074610 con 0x7f3a380d1c80 2024-04-26T21:10:32.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.183+0000 7f3a3d200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3a380d5b40 con 0x7f3a380d2060 2024-04-26T21:10:32.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.184+0000 7f3a37e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a381629d0 0x7f3a38160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.184+0000 7f3a37e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a381629d0 0x7f3a38160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48472/0 (socket says 172.21.15.26:48472) 2024-04-26T21:10:32.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.184+0000 7f3a37e00640 1 -- 172.21.15.26:0/2628336002 learned_addr learned my addr 172.21.15.26:0/2628336002 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:32.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.184+0000 7f3a37400640 1 --2- 172.21.15.26:0/2628336002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a380d1c80 0x7f3a38154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.184+0000 7f3a36a00640 1 --2- 172.21.15.26:0/2628336002 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a380d2060 0x7f3a3815fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.184+0000 7f3a37e00640 1 -- 172.21.15.26:0/2628336002 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a380d2060 msgr2=0x7f3a3815fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.184+0000 7f3a37e00640 1 --2- 172.21.15.26:0/2628336002 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a380d2060 0x7f3a3815fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.184+0000 7f3a37e00640 1 -- 172.21.15.26:0/2628336002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a380d1c80 msgr2=0x7f3a38154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.184+0000 7f3a37e00640 1 --2- 172.21.15.26:0/2628336002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a380d1c80 0x7f3a38154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.184+0000 7f3a37e00640 1 -- 172.21.15.26:0/2628336002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a380b9df0 con 0x7f3a381629d0 2024-04-26T21:10:32.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.184+0000 7f3a37400640 1 --2- 172.21.15.26:0/2628336002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a380d1c80 0x7f3a38154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:10:32.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.185+0000 7f3a37e00640 1 --2- 172.21.15.26:0/2628336002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a381629d0 0x7f3a38160110 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000c780 tx=0x7f3a3000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.185+0000 7f3a2f400640 1 -- 172.21.15.26:0/2628336002 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a30004b60 con 0x7f3a381629d0 2024-04-26T21:10:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.185+0000 7f3a2f400640 1 -- 172.21.15.26:0/2628336002 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a30004d00 con 0x7f3a381629d0 2024-04-26T21:10:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.185+0000 7f3a2f400640 1 -- 172.21.15.26:0/2628336002 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3000aa90 con 0x7f3a381629d0 2024-04-26T21:10:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.185+0000 7f3a3d200640 1 -- 172.21.15.26:0/2628336002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a380ba0d0 con 0x7f3a381629d0 2024-04-26T21:10:32.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.185+0000 7f3a3d200640 1 -- 172.21.15.26:0/2628336002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a380ba690 con 0x7f3a381629d0 2024-04-26T21:10:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.186+0000 7f3a3d200640 1 -- 172.21.15.26:0/2628336002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a3806a700 con 0x7f3a381629d0 2024-04-26T21:10:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.187+0000 7f3a2f400640 1 -- 172.21.15.26:0/2628336002 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a3000ac30 con 0x7f3a381629d0 2024-04-26T21:10:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.188+0000 7f3a2f400640 1 --2- 172.21.15.26:0/2628336002 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3a1806c8f0 0x7f3a1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.188+0000 7f3a2f400640 1 -- 172.21.15.26:0/2628336002 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f3a3008bb70 con 0x7f3a381629d0 2024-04-26T21:10:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.188+0000 7f3a37400640 1 --2- 172.21.15.26:0/2628336002 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3a1806c8f0 0x7f3a1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.189+0000 7f3a37400640 1 --2- 172.21.15.26:0/2628336002 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3a1806c8f0 0x7f3a1806edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3a28006fd0 tx=0x7f3a28008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:32.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.193+0000 7f3a2f400640 1 -- 172.21.15.26:0/2628336002 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3a30020070 con 0x7f3a381629d0 2024-04-26T21:10:32.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.400+0000 7f3a3d200640 1 -- 172.21.15.26:0/2628336002 --> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f3a380cff80 con 0x7f3a1806c8f0 2024-04-26T21:10:32.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.400+0000 7f3a2f400640 1 -- 172.21.15.26:0/2628336002 <== mgr.4104 v2:172.21.15.26:6832/3130430358 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3a380cff80 con 0x7f3a1806c8f0 2024-04-26T21:10:32.402 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 2 to deep-scrub 2024-04-26T21:10:32.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.404+0000 7f3a3d200640 1 -- 172.21.15.26:0/2628336002 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3a1806c8f0 msgr2=0x7f3a1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.404+0000 7f3a3d200640 1 --2- 172.21.15.26:0/2628336002 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3a1806c8f0 0x7f3a1806edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3a28006fd0 tx=0x7f3a28008040 comp rx=0 tx=0).stop 2024-04-26T21:10:32.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.404+0000 7f3a3d200640 1 -- 172.21.15.26:0/2628336002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a381629d0 msgr2=0x7f3a38160110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.404+0000 7f3a3d200640 1 --2- 172.21.15.26:0/2628336002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a381629d0 0x7f3a38160110 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000c780 tx=0x7f3a3000cc40 comp rx=0 tx=0).stop 2024-04-26T21:10:32.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.404+0000 7f3a3d200640 1 -- 172.21.15.26:0/2628336002 shutdown_connections 2024-04-26T21:10:32.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.404+0000 7f3a3d200640 1 --2- 172.21.15.26:0/2628336002 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3a1806c8f0 0x7f3a1806edb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.404+0000 7f3a3d200640 1 --2- 172.21.15.26:0/2628336002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a381629d0 0x7f3a38160110 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.404+0000 7f3a3d200640 1 --2- 172.21.15.26:0/2628336002 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a380d2060 0x7f3a3815fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.404+0000 7f3a3d200640 1 --2- 172.21.15.26:0/2628336002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a380d1c80 0x7f3a38154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.404+0000 7f3a3d200640 1 -- 172.21.15.26:0/2628336002 >> 172.21.15.26:0/2628336002 conn(0x7f3a380d5020 msgr2=0x7f3a38160ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:32.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.404+0000 7f3a3d200640 1 -- 172.21.15.26:0/2628336002 shutdown_connections 2024-04-26T21:10:32.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.404+0000 7f3a3d200640 1 -- 172.21.15.26:0/2628336002 wait complete. 2024-04-26T21:10:32.425 INFO:tasks.ceph:Scrubbing osd.3 2024-04-26T21:10:32.426 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:10:32.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.640+0000 7f7f06c00640 1 Processor -- start 2024-04-26T21:10:32.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.641+0000 7f7f06c00640 1 -- start start 2024-04-26T21:10:32.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.642+0000 7f7f06c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7f0015f950 0x7f7f00073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.642+0000 7f7f06c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f000b4ba0 0x7f7f000b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.642+0000 7f7f06c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f00072f60 con 0x7f7f000d0740 2024-04-26T21:10:32.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.642+0000 7f7f06c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7f001600c0 con 0x7f7f0015f950 2024-04-26T21:10:32.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.642+0000 7f7f06c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7f000d0b20 con 0x7f7f000b4ba0 2024-04-26T21:10:32.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.643+0000 7f7effe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7f000d0740 0x7f7f0015f240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33444/0 (socket says 172.21.15.26:33444) 2024-04-26T21:10:32.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.643+0000 7f7effe00640 1 -- 172.21.15.26:0/808889814 learned_addr learned my addr 172.21.15.26:0/808889814 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:32.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.643+0000 7f7f04c00640 1 --2- 172.21.15.26:0/808889814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f000b4ba0 0x7f7f000b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.643+0000 7f7eff400640 1 --2- 172.21.15.26:0/808889814 >> v2:172.21.15.86:3300/0 conn(0x7f7f0015f950 0x7f7f00073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.643+0000 7f7efea00640 1 -- 172.21.15.26:0/808889814 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1108436154 0 0) 0x7f7f00072f60 con 0x7f7f000d0740 2024-04-26T21:10:32.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.643+0000 7f7efea00640 1 -- 172.21.15.26:0/808889814 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ee4003520 con 0x7f7f000d0740 2024-04-26T21:10:32.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.644+0000 7f7efea00640 1 -- 172.21.15.26:0/808889814 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2549203814 0 0) 0x7f7ee4003520 con 0x7f7f000d0740 2024-04-26T21:10:32.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.644+0000 7f7efea00640 1 -- 172.21.15.26:0/808889814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f000b4ba0 msgr2=0x7f7f000b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:32.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.644+0000 7f7efea00640 1 --2- 172.21.15.26:0/808889814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f000b4ba0 0x7f7f000b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.644+0000 7f7efea00640 1 -- 172.21.15.26:0/808889814 >> v2:172.21.15.86:3300/0 conn(0x7f7f0015f950 msgr2=0x7f7f00073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:32.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.644+0000 7f7efea00640 1 --2- 172.21.15.26:0/808889814 >> v2:172.21.15.86:3300/0 conn(0x7f7f0015f950 0x7f7f00073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.644+0000 7f7efea00640 1 -- 172.21.15.26:0/808889814 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f000731d0 con 0x7f7f000d0740 2024-04-26T21:10:32.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.644+0000 7f7efea00640 1 -- 172.21.15.26:0/808889814 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f7ef0001670 con 0x7f7f000d0740 2024-04-26T21:10:32.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.644+0000 7f7efea00640 1 -- 172.21.15.26:0/808889814 >> v1:172.21.15.26:6789/0 conn(0x7f7f000d0740 legacy=0x7f7f0015f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.644+0000 7f7efea00640 1 --2- 172.21.15.26:0/808889814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee40039b0 0x7f7ee4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.644+0000 7f7efea00640 1 --2- 172.21.15.26:0/808889814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ee4004800 0x7f7ee4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.644+0000 7f7efea00640 1 --2- 172.21.15.26:0/808889814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f000b4ba0 0x7f7ee40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.644+0000 7f7efea00640 1 -- 172.21.15.26:0/808889814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7f000d0b20 con 0x7f7ee4004800 2024-04-26T21:10:32.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.644+0000 7f7efea00640 1 -- 172.21.15.26:0/808889814 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee400b9c0 con 0x7f7ee40039b0 2024-04-26T21:10:32.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.644+0000 7f7efea00640 1 -- 172.21.15.26:0/808889814 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee400dda0 con 0x7f7f000b4ba0 2024-04-26T21:10:32.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.644+0000 7f7eff400640 1 --2- 172.21.15.26:0/808889814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ee4004800 0x7f7ee4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.645+0000 7f7f04c00640 1 --2- 172.21.15.26:0/808889814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f000b4ba0 0x7f7ee40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.645+0000 7f7effe00640 1 --2- 172.21.15.26:0/808889814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee40039b0 0x7f7ee4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.645+0000 7f7eff400640 1 -- 172.21.15.26:0/808889814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f000b4ba0 msgr2=0x7f7ee40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:32.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.645+0000 7f7eff400640 1 --2- 172.21.15.26:0/808889814 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f000b4ba0 0x7f7ee40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.645+0000 7f7eff400640 1 -- 172.21.15.26:0/808889814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee40039b0 msgr2=0x7f7ee4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:32.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.645+0000 7f7eff400640 1 --2- 172.21.15.26:0/808889814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee40039b0 0x7f7ee4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.645+0000 7f7eff400640 1 -- 172.21.15.26:0/808889814 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee4003520 con 0x7f7ee4004800 2024-04-26T21:10:32.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.645+0000 7f7effe00640 1 --2- 172.21.15.26:0/808889814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ee40039b0 0x7f7ee4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:32.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.646+0000 7f7eff400640 1 --2- 172.21.15.26:0/808889814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ee4004800 0x7f7ee4004c80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009af0 tx=0x7f7ee4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5195b51b32ecae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:32.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.646+0000 7f7efea00640 1 -- 172.21.15.26:0/808889814 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec036070 con 0x7f7ee4004800 2024-04-26T21:10:32.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.646+0000 7f7efea00640 1 -- 172.21.15.26:0/808889814 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eec031400 con 0x7f7ee4004800 2024-04-26T21:10:32.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.646+0000 7f7efea00640 1 -- 172.21.15.26:0/808889814 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec009690 con 0x7f7ee4004800 2024-04-26T21:10:32.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.647+0000 7f7f06c00640 1 -- 172.21.15.26:0/808889814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ee4004800 msgr2=0x7f7ee4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.647+0000 7f7f06c00640 1 --2- 172.21.15.26:0/808889814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ee4004800 0x7f7ee4004c80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009af0 tx=0x7f7ee4005830 comp rx=0 tx=0).stop 2024-04-26T21:10:32.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.647+0000 7f7f04c00640 1 -- 172.21.15.26:0/808889814 reap_dead start 2024-04-26T21:10:32.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.647+0000 7f7f06c00640 1 -- 172.21.15.26:0/808889814 shutdown_connections 2024-04-26T21:10:32.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.647+0000 7f7f06c00640 1 -- 172.21.15.26:0/808889814 >> 172.21.15.26:0/808889814 conn(0x7f7f0006c420 msgr2=0x7f7f0006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:32.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.647+0000 7f7f06c00640 1 -- 172.21.15.26:0/808889814 shutdown_connections 2024-04-26T21:10:32.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.647+0000 7f7f06c00640 1 -- 172.21.15.26:0/808889814 wait complete. 2024-04-26T21:10:32.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.650+0000 7f7f06c00640 1 Processor -- start 2024-04-26T21:10:32.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.650+0000 7f7f06c00640 1 -- start start 2024-04-26T21:10:32.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.650+0000 7f7f06c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ee4004800 0x7f7f001498c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.650+0000 7f7f06c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f000b2d00 0x7f7f000b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.650+0000 7f7f06c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f001f9700 0x7f7f001f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.650+0000 7f7f06c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7f00073a70 con 0x7f7ee4004800 2024-04-26T21:10:32.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.650+0000 7f7f06c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f00073790 con 0x7f7f001f9700 2024-04-26T21:10:32.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.650+0000 7f7f06c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7f000b4fd0 con 0x7f7f000b2d00 2024-04-26T21:10:32.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.650+0000 7f7effe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ee4004800 0x7f7f001498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.650+0000 7f7effe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ee4004800 0x7f7f001498c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48494/0 (socket says 172.21.15.26:48494) 2024-04-26T21:10:32.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.650+0000 7f7effe00640 1 -- 172.21.15.26:0/2038554675 learned_addr learned my addr 172.21.15.26:0/2038554675 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:32.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.650+0000 7f7eff400640 1 --2- 172.21.15.26:0/2038554675 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f000b2d00 0x7f7f000b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.651+0000 7f7effe00640 1 -- 172.21.15.26:0/2038554675 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f000b2d00 msgr2=0x7f7f000b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.651+0000 7f7effe00640 1 --2- 172.21.15.26:0/2038554675 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f000b2d00 0x7f7f000b3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.651+0000 7f7effe00640 1 -- 172.21.15.26:0/2038554675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f001f9700 msgr2=0x7f7f001f9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.651+0000 7f7effe00640 1 --2- 172.21.15.26:0/2038554675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f001f9700 0x7f7f001f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.651+0000 7f7effe00640 1 -- 172.21.15.26:0/2038554675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f00158270 con 0x7f7ee4004800 2024-04-26T21:10:32.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.651+0000 7f7effe00640 1 --2- 172.21.15.26:0/2038554675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ee4004800 0x7f7f001498c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0001200 tx=0x7f7ef000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:32.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.651+0000 7f7ef7e00640 1 -- 172.21.15.26:0/2038554675 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0019070 con 0x7f7ee4004800 2024-04-26T21:10:32.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.651+0000 7f7ef7e00640 1 -- 172.21.15.26:0/2038554675 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef00074e0 con 0x7f7ee4004800 2024-04-26T21:10:32.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.651+0000 7f7f06c00640 1 -- 172.21.15.26:0/2038554675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f001fc950 con 0x7f7ee4004800 2024-04-26T21:10:32.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.651+0000 7f7ef7e00640 1 -- 172.21.15.26:0/2038554675 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef00049e0 con 0x7f7ee4004800 2024-04-26T21:10:32.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.651+0000 7f7f06c00640 1 -- 172.21.15.26:0/2038554675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f001fced0 con 0x7f7ee4004800 2024-04-26T21:10:32.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.652+0000 7f7f06c00640 1 -- 172.21.15.26:0/2038554675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7f0006a700 con 0x7f7ee4004800 2024-04-26T21:10:32.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.653+0000 7f7ef7e00640 1 -- 172.21.15.26:0/2038554675 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ef001c070 con 0x7f7ee4004800 2024-04-26T21:10:32.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.654+0000 7f7ef7e00640 1 --2- 172.21.15.26:0/2038554675 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f7ee006c8f0 0x7f7ee006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.654+0000 7f7ef7e00640 1 -- 172.21.15.26:0/2038554675 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f7ef008b340 con 0x7f7ee4004800 2024-04-26T21:10:32.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.655+0000 7f7eff400640 1 --2- 172.21.15.26:0/2038554675 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f7ee006c8f0 0x7f7ee006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.655+0000 7f7ef7e00640 1 --2- 172.21.15.26:0/2038554675 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x7f7ee00727b0 0x7f7ee0074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.655+0000 7f7ef7e00640 1 -- 172.21.15.26:0/2038554675 --> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ee0075280 con 0x7f7ee00727b0 2024-04-26T21:10:32.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.655+0000 7f7ef7e00640 1 -- 172.21.15.26:0/2038554675 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7ef0092050 con 0x7f7ee4004800 2024-04-26T21:10:32.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.655+0000 7f7f04c00640 1 --2- 172.21.15.26:0/2038554675 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x7f7ee00727b0 0x7f7ee0074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.655+0000 7f7eff400640 1 --2- 172.21.15.26:0/2038554675 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f7ee006c8f0 0x7f7ee006edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7eec033000 tx=0x7f7eec008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:32.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.656+0000 7f7f04c00640 1 --2- 172.21.15.26:0/2038554675 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x7f7ee00727b0 0x7f7ee0074bd0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:32.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.656+0000 7f7ef7e00640 1 -- 172.21.15.26:0/2038554675 <== osd.3 v2:172.21.15.26:6809/1280758061 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ee0075280 con 0x7f7ee00727b0 2024-04-26T21:10:32.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.671+0000 7f7f06c00640 1 -- 172.21.15.26:0/2038554675 --> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7f00149e00 con 0x7f7ee00727b0 2024-04-26T21:10:32.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.672+0000 7f7ef7e00640 1 -- 172.21.15.26:0/2038554675 <== osd.3 v2:172.21.15.26:6809/1280758061 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7f00149e00 con 0x7f7ee00727b0 2024-04-26T21:10:32.674 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-26T21:10:32.674 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:10:32.674 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-26T21:10:32.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.672+0000 7f7f06c00640 1 -- 172.21.15.26:0/2038554675 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x7f7ee00727b0 msgr2=0x7f7ee0074bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.672+0000 7f7f06c00640 1 --2- 172.21.15.26:0/2038554675 >> [v2:172.21.15.26:6809/1280758061,v1:172.21.15.26:6810/1280758061] conn(0x7f7ee00727b0 0x7f7ee0074bd0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.673+0000 7f7f06c00640 1 -- 172.21.15.26:0/2038554675 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f7ee006c8f0 msgr2=0x7f7ee006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.673+0000 7f7f06c00640 1 --2- 172.21.15.26:0/2038554675 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f7ee006c8f0 0x7f7ee006edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7eec033000 tx=0x7f7eec008030 comp rx=0 tx=0).stop 2024-04-26T21:10:32.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.673+0000 7f7f06c00640 1 -- 172.21.15.26:0/2038554675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ee4004800 msgr2=0x7f7f001498c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.673+0000 7f7f06c00640 1 --2- 172.21.15.26:0/2038554675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ee4004800 0x7f7f001498c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0001200 tx=0x7f7ef000dc90 comp rx=0 tx=0).stop 2024-04-26T21:10:32.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.673+0000 7f7f04c00640 1 -- 172.21.15.26:0/2038554675 reap_dead start 2024-04-26T21:10:32.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.673+0000 7f7f06c00640 1 -- 172.21.15.26:0/2038554675 shutdown_connections 2024-04-26T21:10:32.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.673+0000 7f7f06c00640 1 -- 172.21.15.26:0/2038554675 >> 172.21.15.26:0/2038554675 conn(0x7f7f0006c420 msgr2=0x7f7f00155720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:32.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.673+0000 7f7f06c00640 1 -- 172.21.15.26:0/2038554675 shutdown_connections 2024-04-26T21:10:32.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.673+0000 7f7f06c00640 1 -- 172.21.15.26:0/2038554675 wait complete. 2024-04-26T21:10:32.696 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-26T21:10:32.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.906+0000 7f88f3000640 1 Processor -- start 2024-04-26T21:10:32.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.906+0000 7f88f3000640 1 -- start start 2024-04-26T21:10:32.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.908+0000 7f88f3000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f88ec1458d0 0x7f88ec073360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.908+0000 7f88f3000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f88ec0738a0 0x7f88ec0d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.908+0000 7f88f3000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88ec0d1230 0x7f88ec06f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.908+0000 7f88f3000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f88ec0d1670 con 0x7f88ec1458d0 2024-04-26T21:10:32.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.908+0000 7f88f3000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f88ec15fd70 con 0x7f88ec0738a0 2024-04-26T21:10:32.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.908+0000 7f88f3000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f88ec160050 con 0x7f88ec0d1230 2024-04-26T21:10:32.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.908+0000 7f88ebe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f88ec1458d0 0x7f88ec073360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.908+0000 7f88ebe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f88ec1458d0 0x7f88ec073360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48504/0 (socket says 172.21.15.26:48504) 2024-04-26T21:10:32.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.908+0000 7f88ebe00640 1 -- 172.21.15.26:0/1025417291 learned_addr learned my addr 172.21.15.26:0/1025417291 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:32.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.908+0000 7f88ebe00640 1 -- 172.21.15.26:0/1025417291 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88ec0d1230 msgr2=0x7f88ec06f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:32.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.908+0000 7f88ebe00640 1 --2- 172.21.15.26:0/1025417291 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88ec0d1230 0x7f88ec06f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.908+0000 7f88e3e00640 1 --2- 172.21.15.26:0/1025417291 >> v2:172.21.15.86:3300/0 conn(0x7f88ec0738a0 0x7f88ec0d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.908+0000 7f88ebe00640 1 -- 172.21.15.26:0/1025417291 >> v2:172.21.15.86:3300/0 conn(0x7f88ec0738a0 msgr2=0x7f88ec0d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:32.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.908+0000 7f88ebe00640 1 --2- 172.21.15.26:0/1025417291 >> v2:172.21.15.86:3300/0 conn(0x7f88ec0738a0 0x7f88ec0d0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.908+0000 7f88ebe00640 1 -- 172.21.15.26:0/1025417291 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ec160330 con 0x7f88ec1458d0 2024-04-26T21:10:32.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.909+0000 7f88ebe00640 1 --2- 172.21.15.26:0/1025417291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88ec1458d0 0x7f88ec073360 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f88ec160260 tx=0x7f88dc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2cc16875ec2720 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:32.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.910+0000 7f88eb400640 1 -- 172.21.15.26:0/1025417291 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc037070 con 0x7f88ec1458d0 2024-04-26T21:10:32.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.910+0000 7f88eb400640 1 -- 172.21.15.26:0/1025417291 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88dc032440 con 0x7f88ec1458d0 2024-04-26T21:10:32.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.910+0000 7f88eb400640 1 -- 172.21.15.26:0/1025417291 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc032810 con 0x7f88ec1458d0 2024-04-26T21:10:32.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.910+0000 7f88f3000640 1 -- 172.21.15.26:0/1025417291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88ec1458d0 msgr2=0x7f88ec073360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.910+0000 7f88f3000640 1 --2- 172.21.15.26:0/1025417291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88ec1458d0 0x7f88ec073360 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f88ec160260 tx=0x7f88dc02a1c0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.910+0000 7f88f3000640 1 -- 172.21.15.26:0/1025417291 shutdown_connections 2024-04-26T21:10:32.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.910+0000 7f88f3000640 1 --2- 172.21.15.26:0/1025417291 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88ec0d1230 0x7f88ec06f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.910+0000 7f88f3000640 1 --2- 172.21.15.26:0/1025417291 >> v2:172.21.15.86:3300/0 conn(0x7f88ec0738a0 0x7f88ec0d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.910+0000 7f88f3000640 1 --2- 172.21.15.26:0/1025417291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88ec1458d0 0x7f88ec073360 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.910+0000 7f88f3000640 1 -- 172.21.15.26:0/1025417291 >> 172.21.15.26:0/1025417291 conn(0x7f88ec0cd840 msgr2=0x7f88ec0cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:32.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.910+0000 7f88f3000640 1 -- 172.21.15.26:0/1025417291 shutdown_connections 2024-04-26T21:10:32.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.911+0000 7f88f3000640 1 -- 172.21.15.26:0/1025417291 wait complete. 2024-04-26T21:10:32.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.912+0000 7f88f3000640 1 Processor -- start 2024-04-26T21:10:32.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.912+0000 7f88f3000640 1 -- start start 2024-04-26T21:10:32.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.913+0000 7f88f3000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88ec0738a0 0x7f88ec1e6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.913+0000 7f88f3000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88ec0d1230 0x7f88ec1e7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.913+0000 7f88f3000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88ec1458d0 0x7f88ec1437d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.913+0000 7f88f3000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec145cb0 con 0x7f88ec0d1230 2024-04-26T21:10:32.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.913+0000 7f88f3000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec1608f0 con 0x7f88ec1458d0 2024-04-26T21:10:32.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.913+0000 7f88f3000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f88ec072f90 con 0x7f88ec0738a0 2024-04-26T21:10:32.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.914+0000 7f88e3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88ec0d1230 0x7f88ec1e7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.914+0000 7f88e3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88ec0d1230 0x7f88ec1e7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48520/0 (socket says 172.21.15.26:48520) 2024-04-26T21:10:32.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.914+0000 7f88e3e00640 1 -- 172.21.15.26:0/266582163 learned_addr learned my addr 172.21.15.26:0/266582163 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:32.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.914+0000 7f88ebe00640 1 --2- 172.21.15.26:0/266582163 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88ec0738a0 0x7f88ec1e6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.914+0000 7f88e3e00640 1 -- 172.21.15.26:0/266582163 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88ec0738a0 msgr2=0x7f88ec1e6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.914+0000 7f88e3e00640 1 --2- 172.21.15.26:0/266582163 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88ec0738a0 0x7f88ec1e6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.914+0000 7f88e3e00640 1 -- 172.21.15.26:0/266582163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88ec1458d0 msgr2=0x7f88ec1437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:32.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.914+0000 7f88f1000640 1 --2- 172.21.15.26:0/266582163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88ec1458d0 0x7f88ec1437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.914+0000 7f88e3e00640 1 --2- 172.21.15.26:0/266582163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88ec1458d0 0x7f88ec1437d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:32.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.914+0000 7f88e3e00640 1 -- 172.21.15.26:0/266582163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ec15b9b0 con 0x7f88ec0d1230 2024-04-26T21:10:32.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.914+0000 7f88f1000640 1 --2- 172.21.15.26:0/266582163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88ec1458d0 0x7f88ec1437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:32.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.915+0000 7f88e3e00640 1 --2- 172.21.15.26:0/266582163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88ec0d1230 0x7f88ec1e7b30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f88d400c7f0 tx=0x7f88d400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:32.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.915+0000 7f88e8c00640 1 -- 172.21.15.26:0/266582163 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d4015070 con 0x7f88ec0d1230 2024-04-26T21:10:32.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.915+0000 7f88e8c00640 1 -- 172.21.15.26:0/266582163 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88d4004240 con 0x7f88ec0d1230 2024-04-26T21:10:32.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.915+0000 7f88e8c00640 1 -- 172.21.15.26:0/266582163 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d4007620 con 0x7f88ec0d1230 2024-04-26T21:10:32.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.915+0000 7f88f3000640 1 -- 172.21.15.26:0/266582163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88ec1e6a90 con 0x7f88ec0d1230 2024-04-26T21:10:32.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.915+0000 7f88f3000640 1 -- 172.21.15.26:0/266582163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88ec1e7330 con 0x7f88ec0d1230 2024-04-26T21:10:32.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.916+0000 7f88f3000640 1 -- 172.21.15.26:0/266582163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88ec1e6e80 con 0x7f88ec0d1230 2024-04-26T21:10:32.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.918+0000 7f88e8c00640 1 -- 172.21.15.26:0/266582163 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f88d40079d0 con 0x7f88ec0d1230 2024-04-26T21:10:32.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.918+0000 7f88e8c00640 1 --2- 172.21.15.26:0/266582163 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f88c806c8f0 0x7f88c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:32.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.918+0000 7f88e8c00640 1 -- 172.21.15.26:0/266582163 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f88d408bc70 con 0x7f88ec0d1230 2024-04-26T21:10:32.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.918+0000 7f88ebe00640 1 --2- 172.21.15.26:0/266582163 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f88c806c8f0 0x7f88c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:32.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.919+0000 7f88ebe00640 1 --2- 172.21.15.26:0/266582163 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f88c806c8f0 0x7f88c806edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f88dc034000 tx=0x7f88dc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:32.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:32.923+0000 7f88e8c00640 1 -- 172.21.15.26:0/266582163 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f88d405a390 con 0x7f88ec0d1230 2024-04-26T21:10:33.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.129+0000 7f88f3000640 1 -- 172.21.15.26:0/266582163 --> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f88ec0b6c90 con 0x7f88c806c8f0 2024-04-26T21:10:33.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.130+0000 7f88e8c00640 1 -- 172.21.15.26:0/266582163 <== mgr.4104 v2:172.21.15.26:6832/3130430358 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f88ec0b6c90 con 0x7f88c806c8f0 2024-04-26T21:10:33.132 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 3 to deep-scrub 2024-04-26T21:10:33.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.132+0000 7f88f3000640 1 -- 172.21.15.26:0/266582163 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f88c806c8f0 msgr2=0x7f88c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.132+0000 7f88f3000640 1 --2- 172.21.15.26:0/266582163 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f88c806c8f0 0x7f88c806edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f88dc034000 tx=0x7f88dc02e000 comp rx=0 tx=0).stop 2024-04-26T21:10:33.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.132+0000 7f88f3000640 1 -- 172.21.15.26:0/266582163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88ec0d1230 msgr2=0x7f88ec1e7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.132+0000 7f88f3000640 1 --2- 172.21.15.26:0/266582163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88ec0d1230 0x7f88ec1e7b30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f88d400c7f0 tx=0x7f88d400ccb0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.133+0000 7f88f3000640 1 -- 172.21.15.26:0/266582163 shutdown_connections 2024-04-26T21:10:33.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.133+0000 7f88f3000640 1 --2- 172.21.15.26:0/266582163 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f88c806c8f0 0x7f88c806edb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.133+0000 7f88f3000640 1 --2- 172.21.15.26:0/266582163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88ec1458d0 0x7f88ec1437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.133+0000 7f88f3000640 1 --2- 172.21.15.26:0/266582163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88ec0d1230 0x7f88ec1e7b30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.133+0000 7f88f3000640 1 --2- 172.21.15.26:0/266582163 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88ec0738a0 0x7f88ec1e6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.133+0000 7f88f3000640 1 -- 172.21.15.26:0/266582163 >> 172.21.15.26:0/266582163 conn(0x7f88ec0cd840 msgr2=0x7f88ec0b6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:33.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.133+0000 7f88f3000640 1 -- 172.21.15.26:0/266582163 shutdown_connections 2024-04-26T21:10:33.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.133+0000 7f88f3000640 1 -- 172.21.15.26:0/266582163 wait complete. 2024-04-26T21:10:33.155 INFO:tasks.ceph:Scrubbing osd.4 2024-04-26T21:10:33.155 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:10:33.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.360+0000 7efeb8200640 1 Processor -- start 2024-04-26T21:10:33.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.361+0000 7efeb8200640 1 -- start start 2024-04-26T21:10:33.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.362+0000 7efeb8200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efeb00b3440 0x7efeb0143270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.362+0000 7efeb8200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeb01611c0 con 0x7efeb013fe30 2024-04-26T21:10:33.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.362+0000 7efeb8200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeb01614a0 con 0x7efeb0140210 2024-04-26T21:10:33.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.362+0000 7efeb8200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7efeb01437b0 con 0x7efeb00b3440 2024-04-26T21:10:33.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.362+0000 7efeb5800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efeb013fe30 0x7efeb00b2d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33450/0 (socket says 172.21.15.26:33450) 2024-04-26T21:10:33.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.362+0000 7efeb5800640 1 -- 172.21.15.26:0/2433079971 learned_addr learned my addr 172.21.15.26:0/2433079971 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:33.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.363+0000 7efeb4e00640 1 --2- 172.21.15.26:0/2433079971 >> v2:172.21.15.86:3300/0 conn(0x7efeb00b3440 0x7efeb0143270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:33.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.363+0000 7efeafe00640 1 -- 172.21.15.26:0/2433079971 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2786541803 0 0) 0x7efeb01611c0 con 0x7efeb013fe30 2024-04-26T21:10:33.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.363+0000 7efeafe00640 1 -- 172.21.15.26:0/2433079971 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe98003520 con 0x7efeb013fe30 2024-04-26T21:10:33.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.363+0000 7efeafe00640 1 -- 172.21.15.26:0/2433079971 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2409529354 0 0) 0x7efe98003520 con 0x7efeb013fe30 2024-04-26T21:10:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.363+0000 7efeafe00640 1 -- 172.21.15.26:0/2433079971 >> v2:172.21.15.86:3300/0 conn(0x7efeb00b3440 msgr2=0x7efeb0143270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.363+0000 7efeafe00640 1 --2- 172.21.15.26:0/2433079971 >> v2:172.21.15.86:3300/0 conn(0x7efeb00b3440 0x7efeb0143270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.363+0000 7efeafe00640 1 -- 172.21.15.26:0/2433079971 >> v1:172.21.15.86:6789/0 conn(0x7efeb0140210 legacy=0x7efeb015dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.363+0000 7efeafe00640 1 -- 172.21.15.26:0/2433079971 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb0161780 con 0x7efeb013fe30 2024-04-26T21:10:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.363+0000 7efeafe00640 1 -- 172.21.15.26:0/2433079971 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7efe9c001670 con 0x7efeb013fe30 2024-04-26T21:10:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.363+0000 7efeafe00640 1 -- 172.21.15.26:0/2433079971 >> v1:172.21.15.26:6789/0 conn(0x7efeb013fe30 legacy=0x7efeb00b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.363+0000 7efeafe00640 1 --2- 172.21.15.26:0/2433079971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe980039b0 0x7efe98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.363+0000 7efeafe00640 1 --2- 172.21.15.26:0/2433079971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe98004800 0x7efe98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.364+0000 7efeafe00640 1 --2- 172.21.15.26:0/2433079971 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe980051c0 0x7efe9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.364+0000 7efeafe00640 1 -- 172.21.15.26:0/2433079971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efeb01437b0 con 0x7efe980039b0 2024-04-26T21:10:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.364+0000 7efeafe00640 1 -- 172.21.15.26:0/2433079971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe9800c060 con 0x7efe98004800 2024-04-26T21:10:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.364+0000 7efeafe00640 1 -- 172.21.15.26:0/2433079971 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7efe9800c260 con 0x7efe980051c0 2024-04-26T21:10:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.364+0000 7efeb5800640 1 --2- 172.21.15.26:0/2433079971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe980039b0 0x7efe98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.364+0000 7efeb5800640 1 -- 172.21.15.26:0/2433079971 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe980051c0 msgr2=0x7efe9800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.364+0000 7efeb5800640 1 --2- 172.21.15.26:0/2433079971 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe980051c0 0x7efe9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.364+0000 7efeb4e00640 1 --2- 172.21.15.26:0/2433079971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe98004800 0x7efe98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.364+0000 7efeb5800640 1 -- 172.21.15.26:0/2433079971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe98004800 msgr2=0x7efe98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.364+0000 7efeb5800640 1 --2- 172.21.15.26:0/2433079971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe98004800 0x7efe98004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.364+0000 7efeb5800640 1 -- 172.21.15.26:0/2433079971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe98003520 con 0x7efe980039b0 2024-04-26T21:10:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.364+0000 7efeb6200640 1 -- 172.21.15.26:0/2433079971 reap_dead start 2024-04-26T21:10:33.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.365+0000 7efeb5800640 1 --2- 172.21.15.26:0/2433079971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe980039b0 0x7efe98003dd0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00c240 tx=0x7efe98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3404e5998b22d9c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.366+0000 7efeafe00640 1 -- 172.21.15.26:0/2433079971 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c007cf0 con 0x7efe980039b0 2024-04-26T21:10:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.366+0000 7efeafe00640 1 -- 172.21.15.26:0/2433079971 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe9c0058d0 con 0x7efe980039b0 2024-04-26T21:10:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.366+0000 7efeafe00640 1 -- 172.21.15.26:0/2433079971 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c005d20 con 0x7efe980039b0 2024-04-26T21:10:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.366+0000 7efeb8200640 1 -- 172.21.15.26:0/2433079971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe980039b0 msgr2=0x7efe98003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.366+0000 7efeb8200640 1 --2- 172.21.15.26:0/2433079971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe980039b0 0x7efe98003dd0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00c240 tx=0x7efe98005830 comp rx=0 tx=0).stop 2024-04-26T21:10:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.366+0000 7efeb8200640 1 -- 172.21.15.26:0/2433079971 shutdown_connections 2024-04-26T21:10:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.366+0000 7efeb8200640 1 --2- 172.21.15.26:0/2433079971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe980039b0 0x7efe98003dd0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.366+0000 7efeb8200640 1 -- 172.21.15.26:0/2433079971 >> 172.21.15.26:0/2433079971 conn(0x7efeb006c420 msgr2=0x7efeb006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.366+0000 7efeb8200640 1 -- 172.21.15.26:0/2433079971 shutdown_connections 2024-04-26T21:10:33.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.367+0000 7efeb8200640 1 -- 172.21.15.26:0/2433079971 wait complete. 2024-04-26T21:10:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.369+0000 7efeb8200640 1 Processor -- start 2024-04-26T21:10:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.369+0000 7efeb8200640 1 -- start start 2024-04-26T21:10:33.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.370+0000 7efeb8200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe980039b0 0x7efeb013d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.370+0000 7efeb8200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeb00b5050 0x7efeb00b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.370+0000 7efeb8200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efeb015f540 0x7efeb015f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.370+0000 7efeb8200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efeb0160d50 con 0x7efe980039b0 2024-04-26T21:10:33.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.370+0000 7efeb8200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efeb00cda70 con 0x7efeb00b5050 2024-04-26T21:10:33.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.370+0000 7efeb8200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7efeb0160ed0 con 0x7efeb015f540 2024-04-26T21:10:33.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.371+0000 7efeb5800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe980039b0 0x7efeb013d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:33.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.371+0000 7efeb5800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe980039b0 0x7efeb013d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48532/0 (socket says 172.21.15.26:48532) 2024-04-26T21:10:33.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.371+0000 7efeb5800640 1 -- 172.21.15.26:0/171062693 learned_addr learned my addr 172.21.15.26:0/171062693 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:33.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.371+0000 7efeb5800640 1 -- 172.21.15.26:0/171062693 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efeb015f540 msgr2=0x7efeb015f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.371+0000 7efeb4e00640 1 --2- 172.21.15.26:0/171062693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeb00b5050 0x7efeb00b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:33.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.371+0000 7efeb5800640 1 --2- 172.21.15.26:0/171062693 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efeb015f540 0x7efeb015f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.371+0000 7efeb5800640 1 -- 172.21.15.26:0/171062693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeb00b5050 msgr2=0x7efeb00b54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.371+0000 7efeb5800640 1 --2- 172.21.15.26:0/171062693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efeb00b5050 0x7efeb00b54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.371+0000 7efeb5800640 1 -- 172.21.15.26:0/171062693 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb01568f0 con 0x7efe980039b0 2024-04-26T21:10:33.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.371+0000 7efeb5800640 1 --2- 172.21.15.26:0/171062693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe980039b0 0x7efeb013d2a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7efe9c001970 tx=0x7efeb0161950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:33.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.372+0000 7efead600640 1 -- 172.21.15.26:0/171062693 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c001c30 con 0x7efe980039b0 2024-04-26T21:10:33.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.372+0000 7efead600640 1 -- 172.21.15.26:0/171062693 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe9c000ed0 con 0x7efe980039b0 2024-04-26T21:10:33.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.372+0000 7efead600640 1 -- 172.21.15.26:0/171062693 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c0049f0 con 0x7efe980039b0 2024-04-26T21:10:33.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.372+0000 7efeb8200640 1 -- 172.21.15.26:0/171062693 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb0156bd0 con 0x7efe980039b0 2024-04-26T21:10:33.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.372+0000 7efeb8200640 1 -- 172.21.15.26:0/171062693 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb0157190 con 0x7efe980039b0 2024-04-26T21:10:33.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.373+0000 7efeb8200640 1 -- 172.21.15.26:0/171062693 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efeb006a700 con 0x7efe980039b0 2024-04-26T21:10:33.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.374+0000 7efead600640 1 -- 172.21.15.26:0/171062693 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe9c03b070 con 0x7efe980039b0 2024-04-26T21:10:33.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.375+0000 7efead600640 1 --2- 172.21.15.26:0/171062693 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7efe9006c940 0x7efe9006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.375+0000 7efead600640 1 -- 172.21.15.26:0/171062693 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7efe9c0ab880 con 0x7efe980039b0 2024-04-26T21:10:33.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.375+0000 7efead600640 1 --2- 172.21.15.26:0/171062693 >> [v2:172.21.15.86:6816/1450967413,v1:172.21.15.86:6817/1450967413] conn(0x7efe90072800 0x7efe90074c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.376+0000 7efead600640 1 -- 172.21.15.26:0/171062693 --> [v2:172.21.15.86:6816/1450967413,v1:172.21.15.86:6817/1450967413] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efe9c009eb0 con 0x7efe90072800 2024-04-26T21:10:33.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.376+0000 7efead600640 1 -- 172.21.15.26:0/171062693 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7efe9c0b2050 con 0x7efe980039b0 2024-04-26T21:10:33.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.376+0000 7efeb4e00640 1 --2- 172.21.15.26:0/171062693 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7efe9006c940 0x7efe9006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:33.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.376+0000 7efeb6200640 1 --2- 172.21.15.26:0/171062693 >> [v2:172.21.15.86:6816/1450967413,v1:172.21.15.86:6817/1450967413] conn(0x7efe90072800 0x7efe90074c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:33.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.376+0000 7efeb4e00640 1 --2- 172.21.15.26:0/171062693 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7efe9006c940 0x7efe9006ee00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efeb013de50 tx=0x7efea4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:33.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.377+0000 7efeb6200640 1 --2- 172.21.15.26:0/171062693 >> [v2:172.21.15.86:6816/1450967413,v1:172.21.15.86:6817/1450967413] conn(0x7efe90072800 0x7efe90074c20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:33.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.379+0000 7efead600640 1 -- 172.21.15.26:0/171062693 <== osd.4 v2:172.21.15.86:6816/1450967413 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe9c009eb0 con 0x7efe90072800 2024-04-26T21:10:33.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.394+0000 7efeb8200640 1 -- 172.21.15.26:0/171062693 --> [v2:172.21.15.86:6816/1450967413,v1:172.21.15.86:6817/1450967413] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7efeb006fe20 con 0x7efe90072800 2024-04-26T21:10:33.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.395+0000 7efead600640 1 -- 172.21.15.26:0/171062693 <== osd.4 v2:172.21.15.86:6816/1450967413 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7efeb006fe20 con 0x7efe90072800 2024-04-26T21:10:33.396 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-26T21:10:33.396 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:10:33.396 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-26T21:10:33.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.396+0000 7efeb8200640 1 -- 172.21.15.26:0/171062693 >> [v2:172.21.15.86:6816/1450967413,v1:172.21.15.86:6817/1450967413] conn(0x7efe90072800 msgr2=0x7efe90074c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.396+0000 7efeb8200640 1 --2- 172.21.15.26:0/171062693 >> [v2:172.21.15.86:6816/1450967413,v1:172.21.15.86:6817/1450967413] conn(0x7efe90072800 0x7efe90074c20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.396+0000 7efeb8200640 1 -- 172.21.15.26:0/171062693 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7efe9006c940 msgr2=0x7efe9006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.396+0000 7efeb8200640 1 --2- 172.21.15.26:0/171062693 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7efe9006c940 0x7efe9006ee00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efeb013de50 tx=0x7efea4008040 comp rx=0 tx=0).stop 2024-04-26T21:10:33.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.396+0000 7efeb8200640 1 -- 172.21.15.26:0/171062693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe980039b0 msgr2=0x7efeb013d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.396+0000 7efeb8200640 1 --2- 172.21.15.26:0/171062693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe980039b0 0x7efeb013d2a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7efe9c001970 tx=0x7efeb0161950 comp rx=0 tx=0).stop 2024-04-26T21:10:33.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.396+0000 7efeb6200640 1 -- 172.21.15.26:0/171062693 reap_dead start 2024-04-26T21:10:33.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.396+0000 7efeb8200640 1 -- 172.21.15.26:0/171062693 shutdown_connections 2024-04-26T21:10:33.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.396+0000 7efeb8200640 1 -- 172.21.15.26:0/171062693 >> 172.21.15.26:0/171062693 conn(0x7efeb006c420 msgr2=0x7efeb00cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:33.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.397+0000 7efeb8200640 1 -- 172.21.15.26:0/171062693 shutdown_connections 2024-04-26T21:10:33.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.397+0000 7efeb8200640 1 -- 172.21.15.26:0/171062693 wait complete. 2024-04-26T21:10:33.414 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-26T21:10:33.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.630+0000 7f94bfe00640 1 Processor -- start 2024-04-26T21:10:33.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.631+0000 7f94bfe00640 1 -- start start 2024-04-26T21:10:33.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.632+0000 7f94bfe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f94c0159b50 0x7f94c015f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.632+0000 7f94bfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f94c015fdd0 0x7f94c00d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.632+0000 7f94bfe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c0160440 con 0x7f94c0159770 2024-04-26T21:10:33.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.632+0000 7f94bfe00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f94c0160200 con 0x7f94c0159b50 2024-04-26T21:10:33.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.632+0000 7f94bfe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f94c0160720 con 0x7f94c015fdd0 2024-04-26T21:10:33.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.633+0000 7f94be000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f94c0159b50 0x7f94c015f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:33.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.633+0000 7f94be000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f94c0159b50 0x7f94c015f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48544/0 (socket says 172.21.15.26:48544) 2024-04-26T21:10:33.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.633+0000 7f94be000640 1 -- 172.21.15.26:0/4196242887 learned_addr learned my addr 172.21.15.26:0/4196242887 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:33.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.633+0000 7f94bf400640 1 --2- 172.21.15.26:0/4196242887 >> v2:172.21.15.86:3300/0 conn(0x7f94c015fdd0 0x7f94c00d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:33.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.633+0000 7f94be000640 1 -- 172.21.15.26:0/4196242887 >> v2:172.21.15.86:3300/0 conn(0x7f94c015fdd0 msgr2=0x7f94c00d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:33.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.633+0000 7f94be000640 1 --2- 172.21.15.26:0/4196242887 >> v2:172.21.15.86:3300/0 conn(0x7f94c015fdd0 0x7f94c00d1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.633+0000 7f94be000640 1 -- 172.21.15.26:0/4196242887 >> v1:172.21.15.86:6789/0 conn(0x7f94c0159770 legacy=0x7f94c00b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:33.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.633+0000 7f94be000640 1 -- 172.21.15.26:0/4196242887 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c0072560 con 0x7f94c0159b50 2024-04-26T21:10:33.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.635+0000 7f94be000640 1 --2- 172.21.15.26:0/4196242887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94c0159b50 0x7f94c015f890 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f94c006fce0 tx=0x7f94ac02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce3c4a007756798 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:33.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.635+0000 7f94bd600640 1 -- 172.21.15.26:0/4196242887 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac037070 con 0x7f94c0159b50 2024-04-26T21:10:33.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.635+0000 7f94bd600640 1 -- 172.21.15.26:0/4196242887 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94ac032440 con 0x7f94c0159b50 2024-04-26T21:10:33.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.635+0000 7f94bd600640 1 -- 172.21.15.26:0/4196242887 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac032810 con 0x7f94c0159b50 2024-04-26T21:10:33.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.636+0000 7f94bfe00640 1 -- 172.21.15.26:0/4196242887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94c0159b50 msgr2=0x7f94c015f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.636+0000 7f94bfe00640 1 --2- 172.21.15.26:0/4196242887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94c0159b50 0x7f94c015f890 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f94c006fce0 tx=0x7f94ac02a1a0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.636+0000 7f94bfe00640 1 -- 172.21.15.26:0/4196242887 shutdown_connections 2024-04-26T21:10:33.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.636+0000 7f94bfe00640 1 --2- 172.21.15.26:0/4196242887 >> v2:172.21.15.86:3300/0 conn(0x7f94c015fdd0 0x7f94c00d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.636+0000 7f94bfe00640 1 --2- 172.21.15.26:0/4196242887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94c0159b50 0x7f94c015f890 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.636+0000 7f94bfe00640 1 -- 172.21.15.26:0/4196242887 >> 172.21.15.26:0/4196242887 conn(0x7f94c006c690 msgr2=0x7f94c006caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:33.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.636+0000 7f94bfe00640 1 -- 172.21.15.26:0/4196242887 shutdown_connections 2024-04-26T21:10:33.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.636+0000 7f94bfe00640 1 -- 172.21.15.26:0/4196242887 wait complete. 2024-04-26T21:10:33.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.638+0000 7f94bfe00640 1 Processor -- start 2024-04-26T21:10:33.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.638+0000 7f94bfe00640 1 -- start start 2024-04-26T21:10:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.639+0000 7f94bfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94c0159770 0x7f94c0152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.639+0000 7f94bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94c0159b50 0x7f94c0154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.639+0000 7f94bfe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f94c015fdd0 0x7f94c0150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.639+0000 7f94bfe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f94c00d1670 con 0x7f94c0159770 2024-04-26T21:10:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.639+0000 7f94bfe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f94c006ee10 con 0x7f94c0159b50 2024-04-26T21:10:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.639+0000 7f94bfe00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f94c00d17f0 con 0x7f94c015fdd0 2024-04-26T21:10:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.639+0000 7f94bea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94c0159770 0x7f94c0152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.639+0000 7f94bea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94c0159770 0x7f94c0152120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48550/0 (socket says 172.21.15.26:48550) 2024-04-26T21:10:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.639+0000 7f94bea00640 1 -- 172.21.15.26:0/2280148527 learned_addr learned my addr 172.21.15.26:0/2280148527 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:33.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.640+0000 7f94bea00640 1 -- 172.21.15.26:0/2280148527 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f94c015fdd0 msgr2=0x7f94c0150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.640+0000 7f94bea00640 1 --2- 172.21.15.26:0/2280148527 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f94c015fdd0 0x7f94c0150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.640+0000 7f94bea00640 1 -- 172.21.15.26:0/2280148527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94c0159b50 msgr2=0x7f94c0154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.640+0000 7f94bea00640 1 --2- 172.21.15.26:0/2280148527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94c0159b50 0x7f94c0154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.640+0000 7f94bea00640 1 -- 172.21.15.26:0/2280148527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c0153660 con 0x7f94c0159770 2024-04-26T21:10:33.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.640+0000 7f94bea00640 1 --2- 172.21.15.26:0/2280148527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94c0159770 0x7f94c0152120 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f94b000c430 tx=0x7f94b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:33.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.640+0000 7f94b6a00640 1 -- 172.21.15.26:0/2280148527 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b0011790 con 0x7f94c0159770 2024-04-26T21:10:33.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.640+0000 7f94b6a00640 1 -- 172.21.15.26:0/2280148527 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94b0017070 con 0x7f94c0159770 2024-04-26T21:10:33.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.640+0000 7f94b6a00640 1 -- 172.21.15.26:0/2280148527 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b0011bb0 con 0x7f94c0159770 2024-04-26T21:10:33.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.640+0000 7f94bfe00640 1 -- 172.21.15.26:0/2280148527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c0153930 con 0x7f94c0159770 2024-04-26T21:10:33.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.641+0000 7f94bfe00640 1 -- 172.21.15.26:0/2280148527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c0154170 con 0x7f94c0159770 2024-04-26T21:10:33.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.642+0000 7f94bfe00640 1 -- 172.21.15.26:0/2280148527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c0154370 con 0x7f94c0159770 2024-04-26T21:10:33.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.643+0000 7f94b6a00640 1 -- 172.21.15.26:0/2280148527 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94b001c460 con 0x7f94c0159770 2024-04-26T21:10:33.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.644+0000 7f94b6a00640 1 --2- 172.21.15.26:0/2280148527 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f94a006c8f0 0x7f94a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:33.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.644+0000 7f94b6a00640 1 -- 172.21.15.26:0/2280148527 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f94b008b6c0 con 0x7f94c0159770 2024-04-26T21:10:33.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.644+0000 7f94be000640 1 --2- 172.21.15.26:0/2280148527 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f94a006c8f0 0x7f94a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:33.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.645+0000 7f94be000640 1 --2- 172.21.15.26:0/2280148527 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f94a006c8f0 0x7f94a006edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f94ac034000 tx=0x7f94ac02c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:33.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.648+0000 7f94b6a00640 1 -- 172.21.15.26:0/2280148527 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f94c0154370 con 0x7f94c0159770 2024-04-26T21:10:33.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.858+0000 7f94bfe00640 1 -- 172.21.15.26:0/2280148527 --> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f94c0068510 con 0x7f94a006c8f0 2024-04-26T21:10:33.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.859+0000 7f94b6a00640 1 -- 172.21.15.26:0/2280148527 <== mgr.4104 v2:172.21.15.26:6832/3130430358 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f94c0068510 con 0x7f94a006c8f0 2024-04-26T21:10:33.860 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 4 to deep-scrub 2024-04-26T21:10:33.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.861+0000 7f94bfe00640 1 -- 172.21.15.26:0/2280148527 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f94a006c8f0 msgr2=0x7f94a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.862+0000 7f94bfe00640 1 --2- 172.21.15.26:0/2280148527 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f94a006c8f0 0x7f94a006edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f94ac034000 tx=0x7f94ac02c000 comp rx=0 tx=0).stop 2024-04-26T21:10:33.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.862+0000 7f94bfe00640 1 -- 172.21.15.26:0/2280148527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94c0159770 msgr2=0x7f94c0152120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:33.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.862+0000 7f94bfe00640 1 --2- 172.21.15.26:0/2280148527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94c0159770 0x7f94c0152120 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f94b000c430 tx=0x7f94b000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.862+0000 7f94bfe00640 1 -- 172.21.15.26:0/2280148527 shutdown_connections 2024-04-26T21:10:33.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.862+0000 7f94bfe00640 1 --2- 172.21.15.26:0/2280148527 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f94a006c8f0 0x7f94a006edb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.862+0000 7f94bfe00640 1 --2- 172.21.15.26:0/2280148527 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f94c015fdd0 0x7f94c0150a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.862+0000 7f94bfe00640 1 --2- 172.21.15.26:0/2280148527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94c0159b50 0x7f94c0154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.862+0000 7f94bfe00640 1 --2- 172.21.15.26:0/2280148527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94c0159770 0x7f94c0152120 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:33.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.862+0000 7f94bfe00640 1 -- 172.21.15.26:0/2280148527 >> 172.21.15.26:0/2280148527 conn(0x7f94c006c690 msgr2=0x7f94c00b5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:33.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.862+0000 7f94bfe00640 1 -- 172.21.15.26:0/2280148527 shutdown_connections 2024-04-26T21:10:33.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:33.862+0000 7f94bfe00640 1 -- 172.21.15.26:0/2280148527 wait complete. 2024-04-26T21:10:33.884 INFO:tasks.ceph:Scrubbing osd.5 2024-04-26T21:10:33.884 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:10:34.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.099+0000 7fdbd4c00640 1 Processor -- start 2024-04-26T21:10:34.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.099+0000 7fdbd4c00640 1 -- start start 2024-04-26T21:10:34.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.101+0000 7fdbd4c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdbd013ff90 0x7fdbd006c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.101+0000 7fdbd4c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdbd006c780 0x7fdbd0160d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.101+0000 7fdbd4c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdbd0161290 0x7fdbd00731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.101+0000 7fdbd4c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fdbd0161ac0 con 0x7fdbd013ff90 2024-04-26T21:10:34.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.101+0000 7fdbd4c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdbd0140370 con 0x7fdbd0161290 2024-04-26T21:10:34.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.101+0000 7fdbd4c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd01435e0 con 0x7fdbd006c780 2024-04-26T21:10:34.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.101+0000 7fdbcde00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdbd013ff90 0x7fdbd006c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.102+0000 7fdbcde00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdbd013ff90 0x7fdbd006c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48558/0 (socket says 172.21.15.26:48558) 2024-04-26T21:10:34.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.102+0000 7fdbcde00640 1 -- 172.21.15.26:0/121069781 learned_addr learned my addr 172.21.15.26:0/121069781 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:34.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.102+0000 7fdbcd400640 1 --2- 172.21.15.26:0/121069781 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdbd006c780 0x7fdbd0160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.102+0000 7fdbcde00640 1 -- 172.21.15.26:0/121069781 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdbd006c780 msgr2=0x7fdbd0160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:34.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.102+0000 7fdbce800640 1 --2- 172.21.15.26:0/121069781 >> v2:172.21.15.86:3300/0 conn(0x7fdbd0161290 0x7fdbd00731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.102+0000 7fdbcde00640 1 --2- 172.21.15.26:0/121069781 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdbd006c780 0x7fdbd0160d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.102+0000 7fdbcde00640 1 -- 172.21.15.26:0/121069781 >> v2:172.21.15.86:3300/0 conn(0x7fdbd0161290 msgr2=0x7fdbd00731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:34.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.102+0000 7fdbcde00640 1 --2- 172.21.15.26:0/121069781 >> v2:172.21.15.86:3300/0 conn(0x7fdbd0161290 0x7fdbd00731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.102+0000 7fdbcde00640 1 -- 172.21.15.26:0/121069781 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd0142820 con 0x7fdbd013ff90 2024-04-26T21:10:34.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.102+0000 7fdbcd400640 1 --2- 172.21.15.26:0/121069781 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdbd006c780 0x7fdbd0160d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:34.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.102+0000 7fdbce800640 1 --2- 172.21.15.26:0/121069781 >> v2:172.21.15.86:3300/0 conn(0x7fdbd0161290 0x7fdbd00731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:34.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.103+0000 7fdbcde00640 1 --2- 172.21.15.26:0/121069781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd013ff90 0x7fdbd006c240 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdbd006cfb0 tx=0x7fdbc002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19a5859b1d60953 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:34.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.103+0000 7fdbcca00640 1 -- 172.21.15.26:0/121069781 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0037070 con 0x7fdbd013ff90 2024-04-26T21:10:34.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.103+0000 7fdbcca00640 1 -- 172.21.15.26:0/121069781 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc0032440 con 0x7fdbd013ff90 2024-04-26T21:10:34.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.104+0000 7fdbcca00640 1 -- 172.21.15.26:0/121069781 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0032810 con 0x7fdbd013ff90 2024-04-26T21:10:34.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.104+0000 7fdbd4c00640 1 -- 172.21.15.26:0/121069781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd013ff90 msgr2=0x7fdbd006c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:34.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.104+0000 7fdbd4c00640 1 --2- 172.21.15.26:0/121069781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd013ff90 0x7fdbd006c240 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdbd006cfb0 tx=0x7fdbc002a140 comp rx=0 tx=0).stop 2024-04-26T21:10:34.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.104+0000 7fdbd4c00640 1 -- 172.21.15.26:0/121069781 shutdown_connections 2024-04-26T21:10:34.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.104+0000 7fdbd4c00640 1 --2- 172.21.15.26:0/121069781 >> v2:172.21.15.86:3300/0 conn(0x7fdbd0161290 0x7fdbd00731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.104+0000 7fdbd4c00640 1 --2- 172.21.15.26:0/121069781 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdbd006c780 0x7fdbd0160d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.104+0000 7fdbd4c00640 1 --2- 172.21.15.26:0/121069781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd013ff90 0x7fdbd006c240 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.104+0000 7fdbd4c00640 1 -- 172.21.15.26:0/121069781 >> 172.21.15.26:0/121069781 conn(0x7fdbd00b3be0 msgr2=0x7fdbd00b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:34.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.105+0000 7fdbd4c00640 1 -- 172.21.15.26:0/121069781 shutdown_connections 2024-04-26T21:10:34.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.105+0000 7fdbd4c00640 1 -- 172.21.15.26:0/121069781 wait complete. 2024-04-26T21:10:34.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.106+0000 7fdbd4c00640 1 Processor -- start 2024-04-26T21:10:34.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.107+0000 7fdbd4c00640 1 -- start start 2024-04-26T21:10:34.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.108+0000 7fdbd4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd006c780 0x7fdbd013edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.108+0000 7fdbd4c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd013ff90 0x7fdbd013c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.108+0000 7fdbd4c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdbd0161290 0x7fdbd013c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.108+0000 7fdbd4c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd01430c0 con 0x7fdbd013ff90 2024-04-26T21:10:34.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.108+0000 7fdbd4c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd0142de0 con 0x7fdbd006c780 2024-04-26T21:10:34.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.108+0000 7fdbd4c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd0161700 con 0x7fdbd0161290 2024-04-26T21:10:34.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.108+0000 7fdbcd400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd013ff90 0x7fdbd013c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.108+0000 7fdbcd400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd013ff90 0x7fdbd013c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48568/0 (socket says 172.21.15.26:48568) 2024-04-26T21:10:34.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.108+0000 7fdbcd400640 1 -- 172.21.15.26:0/648781873 learned_addr learned my addr 172.21.15.26:0/648781873 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:34.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.108+0000 7fdbcde00640 1 --2- 172.21.15.26:0/648781873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd006c780 0x7fdbd013edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.108+0000 7fdbce800640 1 --2- 172.21.15.26:0/648781873 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdbd0161290 0x7fdbd013c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.108+0000 7fdbcd400640 1 -- 172.21.15.26:0/648781873 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdbd0161290 msgr2=0x7fdbd013c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.108+0000 7fdbcd400640 1 --2- 172.21.15.26:0/648781873 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdbd0161290 0x7fdbd013c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.108+0000 7fdbcd400640 1 -- 172.21.15.26:0/648781873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd006c780 msgr2=0x7fdbd013edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.108+0000 7fdbcd400640 1 --2- 172.21.15.26:0/648781873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd006c780 0x7fdbd013edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.108+0000 7fdbcd400640 1 -- 172.21.15.26:0/648781873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd0160720 con 0x7fdbd013ff90 2024-04-26T21:10:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.108+0000 7fdbce800640 1 --2- 172.21.15.26:0/648781873 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdbd0161290 0x7fdbd013c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.109+0000 7fdbcd400640 1 --2- 172.21.15.26:0/648781873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd013ff90 0x7fdbd013c3e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00c820 tx=0x7fdbbc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.109+0000 7fdbc6000640 1 -- 172.21.15.26:0/648781873 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc015070 con 0x7fdbd013ff90 2024-04-26T21:10:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.109+0000 7fdbc6000640 1 -- 172.21.15.26:0/648781873 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbbc007a30 con 0x7fdbd013ff90 2024-04-26T21:10:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.109+0000 7fdbc6000640 1 -- 172.21.15.26:0/648781873 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc007e00 con 0x7fdbd013ff90 2024-04-26T21:10:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.109+0000 7fdbd4c00640 1 -- 172.21.15.26:0/648781873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd00af390 con 0x7fdbd013ff90 2024-04-26T21:10:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.109+0000 7fdbd4c00640 1 -- 172.21.15.26:0/648781873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd00af910 con 0x7fdbd013ff90 2024-04-26T21:10:34.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.110+0000 7fdbd4c00640 1 -- 172.21.15.26:0/648781873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdbd0160d50 con 0x7fdbd013ff90 2024-04-26T21:10:34.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.111+0000 7fdbc6000640 1 -- 172.21.15.26:0/648781873 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdbbc018070 con 0x7fdbd013ff90 2024-04-26T21:10:34.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.112+0000 7fdbc6000640 1 --2- 172.21.15.26:0/648781873 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fdbb006c940 0x7fdbb006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.112+0000 7fdbc6000640 1 -- 172.21.15.26:0/648781873 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7fdbbc08b760 con 0x7fdbd013ff90 2024-04-26T21:10:34.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.113+0000 7fdbc6000640 1 --2- 172.21.15.26:0/648781873 >> [v2:172.21.15.86:6800/1459341072,v1:172.21.15.86:6801/1459341072] conn(0x7fdbb0072800 0x7fdbb0074c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.113+0000 7fdbc6000640 1 -- 172.21.15.26:0/648781873 --> [v2:172.21.15.86:6800/1459341072,v1:172.21.15.86:6801/1459341072] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdbb00752d0 con 0x7fdbb0072800 2024-04-26T21:10:34.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.113+0000 7fdbcde00640 1 --2- 172.21.15.26:0/648781873 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fdbb006c940 0x7fdbb006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.113+0000 7fdbc6000640 1 -- 172.21.15.26:0/648781873 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdbbc092050 con 0x7fdbd013ff90 2024-04-26T21:10:34.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.113+0000 7fdbce800640 1 --2- 172.21.15.26:0/648781873 >> [v2:172.21.15.86:6800/1459341072,v1:172.21.15.86:6801/1459341072] conn(0x7fdbb0072800 0x7fdbb0074c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.114+0000 7fdbcde00640 1 --2- 172.21.15.26:0/648781873 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fdbb006c940 0x7fdbb006ee00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0034000 tx=0x7fdbc002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:34.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.114+0000 7fdbce800640 1 --2- 172.21.15.26:0/648781873 >> [v2:172.21.15.86:6800/1459341072,v1:172.21.15.86:6801/1459341072] conn(0x7fdbb0072800 0x7fdbb0074c20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:34.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.116+0000 7fdbc6000640 1 -- 172.21.15.26:0/648781873 <== osd.5 v2:172.21.15.86:6800/1459341072 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdbb00752d0 con 0x7fdbb0072800 2024-04-26T21:10:34.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.131+0000 7fdbd4c00640 1 -- 172.21.15.26:0/648781873 --> [v2:172.21.15.86:6800/1459341072,v1:172.21.15.86:6801/1459341072] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fdbd006a700 con 0x7fdbb0072800 2024-04-26T21:10:34.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.132+0000 7fdbc6000640 1 -- 172.21.15.26:0/648781873 <== osd.5 v2:172.21.15.86:6800/1459341072 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fdbd006a700 con 0x7fdbb0072800 2024-04-26T21:10:34.133 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-26T21:10:34.133 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:10:34.133 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-26T21:10:34.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.133+0000 7fdbd4c00640 1 -- 172.21.15.26:0/648781873 >> [v2:172.21.15.86:6800/1459341072,v1:172.21.15.86:6801/1459341072] conn(0x7fdbb0072800 msgr2=0x7fdbb0074c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:34.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.133+0000 7fdbd4c00640 1 --2- 172.21.15.26:0/648781873 >> [v2:172.21.15.86:6800/1459341072,v1:172.21.15.86:6801/1459341072] conn(0x7fdbb0072800 0x7fdbb0074c20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.133+0000 7fdbd4c00640 1 -- 172.21.15.26:0/648781873 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fdbb006c940 msgr2=0x7fdbb006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:34.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.133+0000 7fdbd4c00640 1 --2- 172.21.15.26:0/648781873 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fdbb006c940 0x7fdbb006ee00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0034000 tx=0x7fdbc002e000 comp rx=0 tx=0).stop 2024-04-26T21:10:34.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.133+0000 7fdbd4c00640 1 -- 172.21.15.26:0/648781873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd013ff90 msgr2=0x7fdbd013c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:34.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.133+0000 7fdbd4c00640 1 --2- 172.21.15.26:0/648781873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd013ff90 0x7fdbd013c3e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00c820 tx=0x7fdbbc00cc20 comp rx=0 tx=0).stop 2024-04-26T21:10:34.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.133+0000 7fdbce800640 1 -- 172.21.15.26:0/648781873 reap_dead start 2024-04-26T21:10:34.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.133+0000 7fdbd4c00640 1 -- 172.21.15.26:0/648781873 shutdown_connections 2024-04-26T21:10:34.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.133+0000 7fdbd4c00640 1 -- 172.21.15.26:0/648781873 >> 172.21.15.26:0/648781873 conn(0x7fdbd00b3be0 msgr2=0x7fdbd01417e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:34.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.133+0000 7fdbd4c00640 1 -- 172.21.15.26:0/648781873 shutdown_connections 2024-04-26T21:10:34.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.133+0000 7fdbd4c00640 1 -- 172.21.15.26:0/648781873 wait complete. 2024-04-26T21:10:34.153 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-26T21:10:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.364+0000 7f4037c00640 1 Processor -- start 2024-04-26T21:10:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.364+0000 7f4037c00640 1 -- start start 2024-04-26T21:10:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.365+0000 7f4037c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4030146990 0x7f40300b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.365+0000 7f4037c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f403006ec20 0x7f40300d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.365+0000 7f4037c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40300b4480 con 0x7f40300d1210 2024-04-26T21:10:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.365+0000 7f4037c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f4030146d70 con 0x7f4030146990 2024-04-26T21:10:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.365+0000 7f4037c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f40300d1640 con 0x7f403006ec20 2024-04-26T21:10:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.366+0000 7f4035200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4030146990 0x7f40300b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.366+0000 7f4035200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4030146990 0x7f40300b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48584/0 (socket says 172.21.15.26:48584) 2024-04-26T21:10:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.366+0000 7f4035200640 1 -- 172.21.15.26:0/1741803252 learned_addr learned my addr 172.21.15.26:0/1741803252 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:34.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.366+0000 7f402fe00640 1 --2- 172.21.15.26:0/1741803252 >> v2:172.21.15.86:3300/0 conn(0x7f403006ec20 0x7f40300d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.366+0000 7f4035200640 1 -- 172.21.15.26:0/1741803252 >> v2:172.21.15.86:3300/0 conn(0x7f403006ec20 msgr2=0x7f40300d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:34.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.366+0000 7f4035200640 1 --2- 172.21.15.26:0/1741803252 >> v2:172.21.15.86:3300/0 conn(0x7f403006ec20 0x7f40300d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.367+0000 7f4035200640 1 -- 172.21.15.26:0/1741803252 >> v1:172.21.15.86:6789/0 conn(0x7f40300d1210 legacy=0x7f4030072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:34.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.367+0000 7f4035200640 1 -- 172.21.15.26:0/1741803252 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4030073b70 con 0x7f4030146990 2024-04-26T21:10:34.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.367+0000 7f402fe00640 1 --2- 172.21.15.26:0/1741803252 >> v2:172.21.15.86:3300/0 conn(0x7f403006ec20 0x7f40300d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:34.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.368+0000 7f4035200640 1 --2- 172.21.15.26:0/1741803252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4030146990 0x7f40300b3f40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f403015b0c0 tx=0x7f402002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98426f2702177dff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.368+0000 7f402f400640 1 -- 172.21.15.26:0/1741803252 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020037070 con 0x7f4030146990 2024-04-26T21:10:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.368+0000 7f402f400640 1 -- 172.21.15.26:0/1741803252 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4020032440 con 0x7f4030146990 2024-04-26T21:10:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.368+0000 7f402f400640 1 -- 172.21.15.26:0/1741803252 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020032810 con 0x7f4030146990 2024-04-26T21:10:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.369+0000 7f4037c00640 1 -- 172.21.15.26:0/1741803252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4030146990 msgr2=0x7f40300b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.369+0000 7f4037c00640 1 --2- 172.21.15.26:0/1741803252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4030146990 0x7f40300b3f40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f403015b0c0 tx=0x7f402002a230 comp rx=0 tx=0).stop 2024-04-26T21:10:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.369+0000 7f4037c00640 1 -- 172.21.15.26:0/1741803252 shutdown_connections 2024-04-26T21:10:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.369+0000 7f4037c00640 1 --2- 172.21.15.26:0/1741803252 >> v2:172.21.15.86:3300/0 conn(0x7f403006ec20 0x7f40300d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.369+0000 7f4037c00640 1 --2- 172.21.15.26:0/1741803252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4030146990 0x7f40300b3f40 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.369+0000 7f4037c00640 1 -- 172.21.15.26:0/1741803252 >> 172.21.15.26:0/1741803252 conn(0x7f40300cd840 msgr2=0x7f40300cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.369+0000 7f4037c00640 1 -- 172.21.15.26:0/1741803252 shutdown_connections 2024-04-26T21:10:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.369+0000 7f4037c00640 1 -- 172.21.15.26:0/1741803252 wait complete. 2024-04-26T21:10:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.371+0000 7f4037c00640 1 Processor -- start 2024-04-26T21:10:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.371+0000 7f4037c00640 1 -- start start 2024-04-26T21:10:34.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.372+0000 7f4037c00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f403006ec20 0x7f40301f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.372+0000 7f4037c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40300d1210 0x7f403015f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.372+0000 7f4037c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4030146990 0x7f40300b6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.372+0000 7f4037c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f403006e420 con 0x7f4030146990 2024-04-26T21:10:34.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.372+0000 7f4037c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4030074130 con 0x7f40300d1210 2024-04-26T21:10:34.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.372+0000 7f4037c00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f403006e700 con 0x7f403006ec20 2024-04-26T21:10:34.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.372+0000 7f4035c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4030146990 0x7f40300b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.372+0000 7f4035c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4030146990 0x7f40300b6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48600/0 (socket says 172.21.15.26:48600) 2024-04-26T21:10:34.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.372+0000 7f4035c00640 1 -- 172.21.15.26:0/345746352 learned_addr learned my addr 172.21.15.26:0/345746352 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:34.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.372+0000 7f402fe00640 1 --2- 172.21.15.26:0/345746352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40300d1210 0x7f403015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.373+0000 7f4035200640 1 --2- 172.21.15.26:0/345746352 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f403006ec20 0x7f40301f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.373+0000 7f4035c00640 1 -- 172.21.15.26:0/345746352 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f403006ec20 msgr2=0x7f40301f4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:34.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.373+0000 7f4035c00640 1 --2- 172.21.15.26:0/345746352 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f403006ec20 0x7f40301f4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.373+0000 7f4035c00640 1 -- 172.21.15.26:0/345746352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40300d1210 msgr2=0x7f403015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:34.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.373+0000 7f4035c00640 1 --2- 172.21.15.26:0/345746352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40300d1210 0x7f403015f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.373+0000 7f4035c00640 1 -- 172.21.15.26:0/345746352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40300cfbc0 con 0x7f4030146990 2024-04-26T21:10:34.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.373+0000 7f4035c00640 1 --2- 172.21.15.26:0/345746352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4030146990 0x7f40300b6340 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f402800c780 tx=0x7f402800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:34.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.373+0000 7f402cc00640 1 -- 172.21.15.26:0/345746352 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4028004b60 con 0x7f4030146990 2024-04-26T21:10:34.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.374+0000 7f402cc00640 1 -- 172.21.15.26:0/345746352 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4028004d00 con 0x7f4030146990 2024-04-26T21:10:34.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.374+0000 7f402cc00640 1 -- 172.21.15.26:0/345746352 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402800aa90 con 0x7f4030146990 2024-04-26T21:10:34.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.374+0000 7f4037c00640 1 -- 172.21.15.26:0/345746352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40300cfea0 con 0x7f4030146990 2024-04-26T21:10:34.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.374+0000 7f4037c00640 1 -- 172.21.15.26:0/345746352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40300d0460 con 0x7f4030146990 2024-04-26T21:10:34.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.375+0000 7f4037c00640 1 -- 172.21.15.26:0/345746352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f403006a700 con 0x7f4030146990 2024-04-26T21:10:34.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.376+0000 7f402cc00640 1 -- 172.21.15.26:0/345746352 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f402800ac30 con 0x7f4030146990 2024-04-26T21:10:34.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.377+0000 7f402cc00640 1 --2- 172.21.15.26:0/345746352 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f400c06c8f0 0x7f400c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.377+0000 7f402cc00640 1 -- 172.21.15.26:0/345746352 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f402808bbf0 con 0x7f4030146990 2024-04-26T21:10:34.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.377+0000 7f4035200640 1 --2- 172.21.15.26:0/345746352 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f400c06c8f0 0x7f400c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.378+0000 7f4035200640 1 --2- 172.21.15.26:0/345746352 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f400c06c8f0 0x7f400c06edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4020034000 tx=0x7f402002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:34.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.381+0000 7f402cc00640 1 -- 172.21.15.26:0/345746352 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4028020070 con 0x7f4030146990 2024-04-26T21:10:34.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.585+0000 7f4037c00640 1 -- 172.21.15.26:0/345746352 --> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f403015ac60 con 0x7f400c06c8f0 2024-04-26T21:10:34.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.586+0000 7f402cc00640 1 -- 172.21.15.26:0/345746352 <== mgr.4104 v2:172.21.15.26:6832/3130430358 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f403015ac60 con 0x7f400c06c8f0 2024-04-26T21:10:34.587 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 5 to deep-scrub 2024-04-26T21:10:34.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.589+0000 7f4037c00640 1 -- 172.21.15.26:0/345746352 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f400c06c8f0 msgr2=0x7f400c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:34.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.589+0000 7f4037c00640 1 --2- 172.21.15.26:0/345746352 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f400c06c8f0 0x7f400c06edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4020034000 tx=0x7f402002e000 comp rx=0 tx=0).stop 2024-04-26T21:10:34.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.589+0000 7f4037c00640 1 -- 172.21.15.26:0/345746352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4030146990 msgr2=0x7f40300b6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:34.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.589+0000 7f4037c00640 1 --2- 172.21.15.26:0/345746352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4030146990 0x7f40300b6340 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f402800c780 tx=0x7f402800cc40 comp rx=0 tx=0).stop 2024-04-26T21:10:34.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.589+0000 7f4037c00640 1 -- 172.21.15.26:0/345746352 shutdown_connections 2024-04-26T21:10:34.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.589+0000 7f4037c00640 1 --2- 172.21.15.26:0/345746352 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f400c06c8f0 0x7f400c06edb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.589+0000 7f4037c00640 1 --2- 172.21.15.26:0/345746352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4030146990 0x7f40300b6340 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.589+0000 7f4037c00640 1 --2- 172.21.15.26:0/345746352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40300d1210 0x7f403015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.589+0000 7f4037c00640 1 --2- 172.21.15.26:0/345746352 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f403006ec20 0x7f40301f4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.589+0000 7f4037c00640 1 -- 172.21.15.26:0/345746352 >> 172.21.15.26:0/345746352 conn(0x7f40300cd840 msgr2=0x7f40300b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:34.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.589+0000 7f4037c00640 1 -- 172.21.15.26:0/345746352 shutdown_connections 2024-04-26T21:10:34.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.589+0000 7f4037c00640 1 -- 172.21.15.26:0/345746352 wait complete. 2024-04-26T21:10:34.611 INFO:tasks.ceph:Scrubbing osd.6 2024-04-26T21:10:34.611 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:10:34.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.830+0000 7f7f83000640 1 Processor -- start 2024-04-26T21:10:34.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.830+0000 7f7f83000640 1 -- start start 2024-04-26T21:10:34.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.832+0000 7f7f83000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f7c145720 0x7f7f7c0cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.832+0000 7f7f83000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7f7c0cf570 0x7f7f7c06e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.832+0000 7f7f83000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f7c0cfa80 con 0x7f7f7c06ea40 2024-04-26T21:10:34.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.832+0000 7f7f83000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7f7c072140 con 0x7f7f7c0cf570 2024-04-26T21:10:34.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.832+0000 7f7f83000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7f7c06ee90 con 0x7f7f7c145720 2024-04-26T21:10:34.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.833+0000 7f7f81200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7f7c0cf570 0x7f7f7c06e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.833+0000 7f7f81200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7f7c0cf570 0x7f7f7c06e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.26:43474/0 (socket says 172.21.15.26:43474) 2024-04-26T21:10:34.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.833+0000 7f7f81200640 1 -- 172.21.15.26:0/2616536502 learned_addr learned my addr 172.21.15.26:0/2616536502 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:34.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.833+0000 7f7f82600640 1 --1- 172.21.15.26:0/2616536502 >> v1:172.21.15.86:6789/0 conn(0x7f7f7c06ea40 0x7f7f7c162ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.26:36092/0 (socket says 172.21.15.26:36092) 2024-04-26T21:10:34.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.833+0000 7f7f81c00640 1 --2- 172.21.15.26:0/2616536502 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f7c145720 0x7f7f7c0cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.834+0000 7f7f81c00640 1 -- 172.21.15.26:0/2616536502 >> v2:172.21.15.86:3300/0 conn(0x7f7f7c0cf570 msgr2=0x7f7f7c06e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:34.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.834+0000 7f7f81c00640 1 --2- 172.21.15.26:0/2616536502 >> v2:172.21.15.86:3300/0 conn(0x7f7f7c0cf570 0x7f7f7c06e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.834+0000 7f7f81c00640 1 -- 172.21.15.26:0/2616536502 >> v1:172.21.15.86:6789/0 conn(0x7f7f7c06ea40 legacy=0x7f7f7c162ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:34.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.834+0000 7f7f81c00640 1 -- 172.21.15.26:0/2616536502 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7c0cfd60 con 0x7f7f7c145720 2024-04-26T21:10:34.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.834+0000 7f7f81200640 1 --2- 172.21.15.26:0/2616536502 >> v2:172.21.15.86:3300/0 conn(0x7f7f7c0cf570 0x7f7f7c06e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:10:34.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.835+0000 7f7f81c00640 1 --2- 172.21.15.26:0/2616536502 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f7c145720 0x7f7f7c0cf030 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7f700097a0 tx=0x7f7f7002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=280f8ac608821ba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:34.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.835+0000 7f7f7be00640 1 -- 172.21.15.26:0/2616536502 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f70037070 con 0x7f7f7c145720 2024-04-26T21:10:34.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.836+0000 7f7f7be00640 1 -- 172.21.15.26:0/2616536502 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f70032440 con 0x7f7f7c145720 2024-04-26T21:10:34.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.836+0000 7f7f7be00640 1 -- 172.21.15.26:0/2616536502 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f70032810 con 0x7f7f7c145720 2024-04-26T21:10:34.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.836+0000 7f7f83000640 1 -- 172.21.15.26:0/2616536502 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f7c145720 msgr2=0x7f7f7c0cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:34.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.836+0000 7f7f83000640 1 --2- 172.21.15.26:0/2616536502 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f7c145720 0x7f7f7c0cf030 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7f700097a0 tx=0x7f7f7002a210 comp rx=0 tx=0).stop 2024-04-26T21:10:34.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.836+0000 7f7f83000640 1 -- 172.21.15.26:0/2616536502 shutdown_connections 2024-04-26T21:10:34.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.836+0000 7f7f83000640 1 --2- 172.21.15.26:0/2616536502 >> v2:172.21.15.86:3300/0 conn(0x7f7f7c0cf570 0x7f7f7c06e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.836+0000 7f7f83000640 1 --2- 172.21.15.26:0/2616536502 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f7c145720 0x7f7f7c0cf030 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.836+0000 7f7f83000640 1 -- 172.21.15.26:0/2616536502 >> 172.21.15.26:0/2616536502 conn(0x7f7f7c0b2a10 msgr2=0x7f7f7c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:34.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.836+0000 7f7f83000640 1 -- 172.21.15.26:0/2616536502 shutdown_connections 2024-04-26T21:10:34.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.836+0000 7f7f83000640 1 -- 172.21.15.26:0/2616536502 wait complete. 2024-04-26T21:10:34.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.838+0000 7f7f83000640 1 Processor -- start 2024-04-26T21:10:34.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.838+0000 7f7f83000640 1 -- start start 2024-04-26T21:10:34.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.839+0000 7f7f83000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f7c06ea40 0x7f7f7c14e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.839+0000 7f7f83000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f7c0cf570 0x7f7f7c141800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.839+0000 7f7f83000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f7c145720 0x7f7f7c0c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.839+0000 7f7f83000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c0d0320 con 0x7f7f7c145720 2024-04-26T21:10:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.839+0000 7f7f83000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c0d0690 con 0x7f7f7c0cf570 2024-04-26T21:10:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.839+0000 7f7f83000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f7f7c145b00 con 0x7f7f7c06ea40 2024-04-26T21:10:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.840+0000 7f7f82600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f7c145720 0x7f7f7c0c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.840+0000 7f7f82600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f7c145720 0x7f7f7c0c5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48606/0 (socket says 172.21.15.26:48606) 2024-04-26T21:10:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.840+0000 7f7f82600640 1 -- 172.21.15.26:0/1302135120 learned_addr learned my addr 172.21.15.26:0/1302135120 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.840+0000 7f7f81c00640 1 --2- 172.21.15.26:0/1302135120 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f7c06ea40 0x7f7f7c14e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.840+0000 7f7f81200640 1 --2- 172.21.15.26:0/1302135120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f7c0cf570 0x7f7f7c141800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.840+0000 7f7f82600640 1 -- 172.21.15.26:0/1302135120 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f7c06ea40 msgr2=0x7f7f7c14e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.840+0000 7f7f82600640 1 --2- 172.21.15.26:0/1302135120 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f7c06ea40 0x7f7f7c14e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.840+0000 7f7f82600640 1 -- 172.21.15.26:0/1302135120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f7c0cf570 msgr2=0x7f7f7c141800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.840+0000 7f7f82600640 1 --2- 172.21.15.26:0/1302135120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f7c0cf570 0x7f7f7c141800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.840+0000 7f7f82600640 1 -- 172.21.15.26:0/1302135120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7c1f5f00 con 0x7f7f7c145720 2024-04-26T21:10:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.840+0000 7f7f81c00640 1 --2- 172.21.15.26:0/1302135120 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f7c06ea40 0x7f7f7c14e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:10:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.841+0000 7f7f82600640 1 --2- 172.21.15.26:0/1302135120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f7c145720 0x7f7f7c0c5e40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400c780 tx=0x7f7f7400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.841+0000 7f7f79600640 1 -- 172.21.15.26:0/1302135120 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74017070 con 0x7f7f7c145720 2024-04-26T21:10:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.841+0000 7f7f79600640 1 -- 172.21.15.26:0/1302135120 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f7400a640 con 0x7f7f7c145720 2024-04-26T21:10:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.841+0000 7f7f79600640 1 -- 172.21.15.26:0/1302135120 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7400aa30 con 0x7f7f7c145720 2024-04-26T21:10:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.841+0000 7f7f83000640 1 -- 172.21.15.26:0/1302135120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f7c1f5280 con 0x7f7f7c145720 2024-04-26T21:10:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.841+0000 7f7f83000640 1 -- 172.21.15.26:0/1302135120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f7c1f5840 con 0x7f7f7c145720 2024-04-26T21:10:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.842+0000 7f7f83000640 1 -- 172.21.15.26:0/1302135120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7f7c06a700 con 0x7f7f7c145720 2024-04-26T21:10:34.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.843+0000 7f7f79600640 1 -- 172.21.15.26:0/1302135120 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f7401a070 con 0x7f7f7c145720 2024-04-26T21:10:34.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.844+0000 7f7f79600640 1 --2- 172.21.15.26:0/1302135120 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f7f5c06c920 0x7f7f5c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.844+0000 7f7f79600640 1 -- 172.21.15.26:0/1302135120 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f7f7408ad80 con 0x7f7f7c145720 2024-04-26T21:10:34.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.845+0000 7f7f79600640 1 --2- 172.21.15.26:0/1302135120 >> [v2:172.21.15.86:6823/4235789402,v1:172.21.15.86:6825/4235789402] conn(0x7f7f5c0727e0 0x7f7f5c074c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:34.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.845+0000 7f7f81c00640 1 --2- 172.21.15.26:0/1302135120 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f7f5c06c920 0x7f7f5c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.845+0000 7f7f79600640 1 -- 172.21.15.26:0/1302135120 --> [v2:172.21.15.86:6823/4235789402,v1:172.21.15.86:6825/4235789402] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7f74020e50 con 0x7f7f5c0727e0 2024-04-26T21:10:34.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.845+0000 7f7f79600640 1 -- 172.21.15.26:0/1302135120 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7f7408f030 con 0x7f7f7c145720 2024-04-26T21:10:34.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.845+0000 7f7f81200640 1 --2- 172.21.15.26:0/1302135120 >> [v2:172.21.15.86:6823/4235789402,v1:172.21.15.86:6825/4235789402] conn(0x7f7f5c0727e0 0x7f7f5c074c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:34.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.845+0000 7f7f81c00640 1 --2- 172.21.15.26:0/1302135120 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f7f5c06c920 0x7f7f5c06ede0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7f70034000 tx=0x7f7f7002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:34.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.846+0000 7f7f81200640 1 --2- 172.21.15.26:0/1302135120 >> [v2:172.21.15.86:6823/4235789402,v1:172.21.15.86:6825/4235789402] conn(0x7f7f5c0727e0 0x7f7f5c074c00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:34.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.848+0000 7f7f79600640 1 -- 172.21.15.26:0/1302135120 <== osd.6 v2:172.21.15.86:6823/4235789402 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f74020e50 con 0x7f7f5c0727e0 2024-04-26T21:10:34.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.863+0000 7f7f83000640 1 -- 172.21.15.26:0/1302135120 --> [v2:172.21.15.86:6823/4235789402,v1:172.21.15.86:6825/4235789402] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7f7c0b3720 con 0x7f7f5c0727e0 2024-04-26T21:10:34.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.864+0000 7f7f79600640 1 -- 172.21.15.26:0/1302135120 <== osd.6 v2:172.21.15.86:6823/4235789402 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7f7c0b3720 con 0x7f7f5c0727e0 2024-04-26T21:10:34.865 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-26T21:10:34.865 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:10:34.865 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-26T21:10:34.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.865+0000 7f7f83000640 1 -- 172.21.15.26:0/1302135120 >> [v2:172.21.15.86:6823/4235789402,v1:172.21.15.86:6825/4235789402] conn(0x7f7f5c0727e0 msgr2=0x7f7f5c074c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:34.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.865+0000 7f7f83000640 1 --2- 172.21.15.26:0/1302135120 >> [v2:172.21.15.86:6823/4235789402,v1:172.21.15.86:6825/4235789402] conn(0x7f7f5c0727e0 0x7f7f5c074c00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:34.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.865+0000 7f7f83000640 1 -- 172.21.15.26:0/1302135120 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f7f5c06c920 msgr2=0x7f7f5c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:34.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.865+0000 7f7f83000640 1 --2- 172.21.15.26:0/1302135120 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f7f5c06c920 0x7f7f5c06ede0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7f70034000 tx=0x7f7f7002e000 comp rx=0 tx=0).stop 2024-04-26T21:10:34.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.865+0000 7f7f83000640 1 -- 172.21.15.26:0/1302135120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f7c145720 msgr2=0x7f7f7c0c5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:34.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.865+0000 7f7f83000640 1 --2- 172.21.15.26:0/1302135120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f7c145720 0x7f7f7c0c5e40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400c780 tx=0x7f7f7400cc40 comp rx=0 tx=0).stop 2024-04-26T21:10:34.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.865+0000 7f7f82600640 1 -- 172.21.15.26:0/1302135120 reap_dead start 2024-04-26T21:10:34.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.865+0000 7f7f83000640 1 -- 172.21.15.26:0/1302135120 shutdown_connections 2024-04-26T21:10:34.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.865+0000 7f7f83000640 1 -- 172.21.15.26:0/1302135120 >> 172.21.15.26:0/1302135120 conn(0x7f7f7c0b2a10 msgr2=0x7f7f7c0b08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:34.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.865+0000 7f7f83000640 1 -- 172.21.15.26:0/1302135120 shutdown_connections 2024-04-26T21:10:34.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:34.865+0000 7f7f83000640 1 -- 172.21.15.26:0/1302135120 wait complete. 2024-04-26T21:10:34.883 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-26T21:10:35.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.080+0000 7fb214a00640 1 Processor -- start 2024-04-26T21:10:35.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.080+0000 7fb214a00640 1 -- start start 2024-04-26T21:10:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.082+0000 7fb214a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb21015f080 0x7fb2100c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.082+0000 7fb214a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb210072eb0 0x7fb21006c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.082+0000 7fb214a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb210163680 con 0x7fb2100c6410 2024-04-26T21:10:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.082+0000 7fb214a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7fb210163920 con 0x7fb210072eb0 2024-04-26T21:10:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.082+0000 7fb214a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb210070110 con 0x7fb21015f080 2024-04-26T21:10:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.082+0000 7fb20de00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb2100c6410 0x7fb2100727a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33452/0 (socket says 172.21.15.26:33452) 2024-04-26T21:10:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.082+0000 7fb20de00640 1 -- 172.21.15.26:0/3204482393 learned_addr learned my addr 172.21.15.26:0/3204482393 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.082+0000 7fb20f200640 1 --2- 172.21.15.26:0/3204482393 >> v2:172.21.15.26:3300/0 conn(0x7fb210072eb0 0x7fb21006c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:35.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.082+0000 7fb20e800640 1 --2- 172.21.15.26:0/3204482393 >> v2:172.21.15.86:3300/0 conn(0x7fb21015f080 0x7fb2100c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.083+0000 7fb20d400640 1 -- 172.21.15.26:0/3204482393 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3023877104 0 0) 0x7fb210163680 con 0x7fb2100c6410 2024-04-26T21:10:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.083+0000 7fb20f200640 1 -- 172.21.15.26:0/3204482393 >> v2:172.21.15.86:3300/0 conn(0x7fb21015f080 msgr2=0x7fb2100c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.083+0000 7fb20f200640 1 --2- 172.21.15.26:0/3204482393 >> v2:172.21.15.86:3300/0 conn(0x7fb21015f080 0x7fb2100c7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.083+0000 7fb20f200640 1 -- 172.21.15.26:0/3204482393 >> v1:172.21.15.26:6789/0 conn(0x7fb2100c6410 legacy=0x7fb2100727a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:35.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.083+0000 7fb20f200640 1 -- 172.21.15.26:0/3204482393 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2100703f0 con 0x7fb210072eb0 2024-04-26T21:10:35.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.083+0000 7fb20e800640 1 --2- 172.21.15.26:0/3204482393 >> v2:172.21.15.86:3300/0 conn(0x7fb21015f080 0x7fb2100c7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:10:35.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.084+0000 7fb20f200640 1 --2- 172.21.15.26:0/3204482393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb210072eb0 0x7fb21006c520 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb210071bc0 tx=0x7fb20802a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166e38d2e2f0bea7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.084+0000 7fb20d400640 1 -- 172.21.15.26:0/3204482393 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb208007740 con 0x7fb210072eb0 2024-04-26T21:10:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.084+0000 7fb20d400640 1 -- 172.21.15.26:0/3204482393 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2080078e0 con 0x7fb210072eb0 2024-04-26T21:10:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.084+0000 7fb20d400640 1 -- 172.21.15.26:0/3204482393 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb208007cb0 con 0x7fb210072eb0 2024-04-26T21:10:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.085+0000 7fb214a00640 1 -- 172.21.15.26:0/3204482393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb210072eb0 msgr2=0x7fb21006c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.085+0000 7fb214a00640 1 --2- 172.21.15.26:0/3204482393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb210072eb0 0x7fb21006c520 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb210071bc0 tx=0x7fb20802a560 comp rx=0 tx=0).stop 2024-04-26T21:10:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.085+0000 7fb214a00640 1 -- 172.21.15.26:0/3204482393 shutdown_connections 2024-04-26T21:10:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.085+0000 7fb214a00640 1 --2- 172.21.15.26:0/3204482393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb210072eb0 0x7fb21006c520 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.085+0000 7fb214a00640 1 --2- 172.21.15.26:0/3204482393 >> v2:172.21.15.86:3300/0 conn(0x7fb21015f080 0x7fb2100c7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.085+0000 7fb214a00640 1 -- 172.21.15.26:0/3204482393 >> 172.21.15.26:0/3204482393 conn(0x7fb2100cded0 msgr2=0x7fb2100ce2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.085+0000 7fb214a00640 1 -- 172.21.15.26:0/3204482393 shutdown_connections 2024-04-26T21:10:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.085+0000 7fb214a00640 1 -- 172.21.15.26:0/3204482393 wait complete. 2024-04-26T21:10:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.087+0000 7fb214a00640 1 Processor -- start 2024-04-26T21:10:35.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.087+0000 7fb214a00640 1 -- start start 2024-04-26T21:10:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.088+0000 7fb214a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb210072eb0 0x7fb210158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.088+0000 7fb214a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2100c6410 0x7fb2101f8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.088+0000 7fb214a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb21015f080 0x7fb2100c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.088+0000 7fb214a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2100c8190 con 0x7fb2100c6410 2024-04-26T21:10:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.088+0000 7fb214a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb21015f460 con 0x7fb210072eb0 2024-04-26T21:10:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.088+0000 7fb214a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fb21006ca60 con 0x7fb21015f080 2024-04-26T21:10:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.088+0000 7fb20de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2100c6410 0x7fb2101f8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.088+0000 7fb20de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2100c6410 0x7fb2101f8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48632/0 (socket says 172.21.15.26:48632) 2024-04-26T21:10:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.088+0000 7fb20de00640 1 -- 172.21.15.26:0/268814478 learned_addr learned my addr 172.21.15.26:0/268814478 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.088+0000 7fb20e800640 1 --2- 172.21.15.26:0/268814478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb210072eb0 0x7fb210158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:35.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.089+0000 7fb20de00640 1 -- 172.21.15.26:0/268814478 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb21015f080 msgr2=0x7fb2100c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:35.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.089+0000 7fb20de00640 1 --2- 172.21.15.26:0/268814478 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb21015f080 0x7fb2100c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:35.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.089+0000 7fb20de00640 1 -- 172.21.15.26:0/268814478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb210072eb0 msgr2=0x7fb210158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:35.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.089+0000 7fb20de00640 1 --2- 172.21.15.26:0/268814478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb210072eb0 0x7fb210158610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:35.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.089+0000 7fb20de00640 1 -- 172.21.15.26:0/268814478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb210156000 con 0x7fb2100c6410 2024-04-26T21:10:35.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.089+0000 7fb20e800640 1 --2- 172.21.15.26:0/268814478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb210072eb0 0x7fb210158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:35.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.089+0000 7fb20de00640 1 --2- 172.21.15.26:0/268814478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2100c6410 0x7fb2101f8550 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800c500 tx=0x7fb1f800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:35.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.089+0000 7fb206a00640 1 -- 172.21.15.26:0/268814478 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8018070 con 0x7fb2100c6410 2024-04-26T21:10:35.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.089+0000 7fb206a00640 1 -- 172.21.15.26:0/268814478 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f8007580 con 0x7fb2100c6410 2024-04-26T21:10:35.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.089+0000 7fb214a00640 1 -- 172.21.15.26:0/268814478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2101562e0 con 0x7fb2100c6410 2024-04-26T21:10:35.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.089+0000 7fb206a00640 1 -- 172.21.15.26:0/268814478 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8004a20 con 0x7fb2100c6410 2024-04-26T21:10:35.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.089+0000 7fb214a00640 1 -- 172.21.15.26:0/268814478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2101f8030 con 0x7fb2100c6410 2024-04-26T21:10:35.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.090+0000 7fb214a00640 1 -- 172.21.15.26:0/268814478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2100693d0 con 0x7fb2100c6410 2024-04-26T21:10:35.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.092+0000 7fb206a00640 1 -- 172.21.15.26:0/268814478 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb1f801b070 con 0x7fb2100c6410 2024-04-26T21:10:35.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.093+0000 7fb206a00640 1 --2- 172.21.15.26:0/268814478 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb1ec06c8f0 0x7fb1ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:35.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.093+0000 7fb206a00640 1 -- 172.21.15.26:0/268814478 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7fb1f808abc0 con 0x7fb2100c6410 2024-04-26T21:10:35.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.093+0000 7fb20e800640 1 --2- 172.21.15.26:0/268814478 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb1ec06c8f0 0x7fb1ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:35.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.094+0000 7fb20e800640 1 --2- 172.21.15.26:0/268814478 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb1ec06c8f0 0x7fb1ec06edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb200006fd0 tx=0x7fb200008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:35.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.098+0000 7fb206a00640 1 -- 172.21.15.26:0/268814478 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb1f80592e0 con 0x7fb2100c6410 2024-04-26T21:10:35.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.306+0000 7fb214a00640 1 -- 172.21.15.26:0/268814478 --> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fb210073990 con 0x7fb1ec06c8f0 2024-04-26T21:10:35.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.306+0000 7fb206a00640 1 -- 172.21.15.26:0/268814478 <== mgr.4104 v2:172.21.15.26:6832/3130430358 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb210073990 con 0x7fb1ec06c8f0 2024-04-26T21:10:35.307 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 6 to deep-scrub 2024-04-26T21:10:35.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.309+0000 7fb214a00640 1 -- 172.21.15.26:0/268814478 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb1ec06c8f0 msgr2=0x7fb1ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:35.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.309+0000 7fb214a00640 1 --2- 172.21.15.26:0/268814478 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb1ec06c8f0 0x7fb1ec06edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb200006fd0 tx=0x7fb200008040 comp rx=0 tx=0).stop 2024-04-26T21:10:35.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.309+0000 7fb214a00640 1 -- 172.21.15.26:0/268814478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2100c6410 msgr2=0x7fb2101f8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:35.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.309+0000 7fb214a00640 1 --2- 172.21.15.26:0/268814478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2100c6410 0x7fb2101f8550 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800c500 tx=0x7fb1f800c9c0 comp rx=0 tx=0).stop 2024-04-26T21:10:35.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.309+0000 7fb214a00640 1 -- 172.21.15.26:0/268814478 shutdown_connections 2024-04-26T21:10:35.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.309+0000 7fb214a00640 1 --2- 172.21.15.26:0/268814478 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fb1ec06c8f0 0x7fb1ec06edb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:35.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.309+0000 7fb214a00640 1 --2- 172.21.15.26:0/268814478 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb21015f080 0x7fb2100c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:35.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.309+0000 7fb214a00640 1 --2- 172.21.15.26:0/268814478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2100c6410 0x7fb2101f8550 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:35.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.309+0000 7fb214a00640 1 --2- 172.21.15.26:0/268814478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb210072eb0 0x7fb210158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:35.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.309+0000 7fb214a00640 1 -- 172.21.15.26:0/268814478 >> 172.21.15.26:0/268814478 conn(0x7fb2100cded0 msgr2=0x7fb210155300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:35.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.310+0000 7fb214a00640 1 -- 172.21.15.26:0/268814478 shutdown_connections 2024-04-26T21:10:35.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.310+0000 7fb214a00640 1 -- 172.21.15.26:0/268814478 wait complete. 2024-04-26T21:10:35.330 INFO:tasks.ceph:Scrubbing osd.7 2024-04-26T21:10:35.330 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:10:35.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.548+0000 7f0a21800640 1 Processor -- start 2024-04-26T21:10:35.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.549+0000 7f0a21800640 1 -- start start 2024-04-26T21:10:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.550+0000 7f0a21800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0a1c162760 0x7f0a1c0b7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.550+0000 7f0a21800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a1c0d1a50 0x7f0a1c0d1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.550+0000 7f0a21800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a1c0b66a0 con 0x7f0a1c162b40 2024-04-26T21:10:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.550+0000 7f0a21800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f0a1c0d4080 con 0x7f0a1c162760 2024-04-26T21:10:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.550+0000 7f0a21800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0a1c0cfc00 con 0x7f0a1c0d1a50 2024-04-26T21:10:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.551+0000 7f0a1b400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0a1c162b40 0x7f0a1c0b5f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33462/0 (socket says 172.21.15.26:33462) 2024-04-26T21:10:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.551+0000 7f0a1b400640 1 -- 172.21.15.26:0/3922846464 learned_addr learned my addr 172.21.15.26:0/3922846464 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.551+0000 7f0a1be00640 1 --2- 172.21.15.26:0/3922846464 >> v2:172.21.15.26:3300/0 conn(0x7f0a1c162760 0x7f0a1c0b7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.551+0000 7f0a20e00640 1 --2- 172.21.15.26:0/3922846464 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a1c0d1a50 0x7f0a1c0d1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.551+0000 7f0a1aa00640 1 -- 172.21.15.26:0/3922846464 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2663717105 0 0) 0x7f0a1c0b66a0 con 0x7f0a1c162b40 2024-04-26T21:10:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.551+0000 7f0a1be00640 1 -- 172.21.15.26:0/3922846464 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a1c0d1a50 msgr2=0x7f0a1c0d1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.551+0000 7f0a1be00640 1 --2- 172.21.15.26:0/3922846464 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a1c0d1a50 0x7f0a1c0d1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.551+0000 7f0a1be00640 1 -- 172.21.15.26:0/3922846464 >> v1:172.21.15.26:6789/0 conn(0x7f0a1c162b40 legacy=0x7f0a1c0b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.551+0000 7f0a1be00640 1 -- 172.21.15.26:0/3922846464 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a1c0d37e0 con 0x7f0a1c162760 2024-04-26T21:10:35.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.552+0000 7f0a1be00640 1 --2- 172.21.15.26:0/3922846464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a1c162760 0x7f0a1c0b7eb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0097d0 tx=0x7f0a0c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66becb9d5559e404 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:35.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.553+0000 7f0a1aa00640 1 -- 172.21.15.26:0/3922846464 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c037070 con 0x7f0a1c162760 2024-04-26T21:10:35.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.553+0000 7f0a1aa00640 1 -- 172.21.15.26:0/3922846464 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a0c02ae40 con 0x7f0a1c162760 2024-04-26T21:10:35.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.553+0000 7f0a1aa00640 1 -- 172.21.15.26:0/3922846464 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c032660 con 0x7f0a1c162760 2024-04-26T21:10:35.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.553+0000 7f0a21800640 1 -- 172.21.15.26:0/3922846464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a1c162760 msgr2=0x7f0a1c0b7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:35.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.553+0000 7f0a21800640 1 --2- 172.21.15.26:0/3922846464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a1c162760 0x7f0a1c0b7eb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0097d0 tx=0x7f0a0c02a130 comp rx=0 tx=0).stop 2024-04-26T21:10:35.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.553+0000 7f0a21800640 1 -- 172.21.15.26:0/3922846464 shutdown_connections 2024-04-26T21:10:35.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.553+0000 7f0a21800640 1 --2- 172.21.15.26:0/3922846464 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a1c0d1a50 0x7f0a1c0d1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:35.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.553+0000 7f0a21800640 1 --2- 172.21.15.26:0/3922846464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a1c162760 0x7f0a1c0b7eb0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:35.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.553+0000 7f0a21800640 1 -- 172.21.15.26:0/3922846464 >> 172.21.15.26:0/3922846464 conn(0x7f0a1c0d5060 msgr2=0x7f0a1c0d5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:35.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.553+0000 7f0a21800640 1 -- 172.21.15.26:0/3922846464 shutdown_connections 2024-04-26T21:10:35.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.553+0000 7f0a21800640 1 -- 172.21.15.26:0/3922846464 wait complete. 2024-04-26T21:10:35.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.555+0000 7f0a21800640 1 Processor -- start 2024-04-26T21:10:35.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.555+0000 7f0a21800640 1 -- start start 2024-04-26T21:10:35.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.556+0000 7f0a21800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a1c0d1a50 0x7f0a1c15c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:35.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.556+0000 7f0a21800640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a1c162760 0x7f0a1c0cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:35.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.556+0000 7f0a21800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a1c162b40 0x7f0a1c0cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:35.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.556+0000 7f0a21800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c0cfea0 con 0x7f0a1c162b40 2024-04-26T21:10:35.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.556+0000 7f0a21800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c0d3da0 con 0x7f0a1c0d1a50 2024-04-26T21:10:35.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.556+0000 7f0a21800640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f0a1c0d5d50 con 0x7f0a1c162760 2024-04-26T21:10:35.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.557+0000 7f0a20e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a1c162b40 0x7f0a1c0cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:35.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.557+0000 7f0a20e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a1c162b40 0x7f0a1c0cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48650/0 (socket says 172.21.15.26:48650) 2024-04-26T21:10:35.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.557+0000 7f0a20e00640 1 -- 172.21.15.26:0/1806369129 learned_addr learned my addr 172.21.15.26:0/1806369129 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:35.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.557+0000 7f0a1be00640 1 --2- 172.21.15.26:0/1806369129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a1c0d1a50 0x7f0a1c15c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:35.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.557+0000 7f0a1b400640 1 --2- 172.21.15.26:0/1806369129 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a1c162760 0x7f0a1c0cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:35.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.557+0000 7f0a20e00640 1 -- 172.21.15.26:0/1806369129 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a1c162760 msgr2=0x7f0a1c0cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.557+0000 7f0a20e00640 1 --2- 172.21.15.26:0/1806369129 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a1c162760 0x7f0a1c0cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.557+0000 7f0a20e00640 1 -- 172.21.15.26:0/1806369129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a1c0d1a50 msgr2=0x7f0a1c15c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.557+0000 7f0a20e00640 1 --2- 172.21.15.26:0/1806369129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a1c0d1a50 0x7f0a1c15c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.557+0000 7f0a20e00640 1 -- 172.21.15.26:0/1806369129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a1c0ca700 con 0x7f0a1c162b40 2024-04-26T21:10:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.558+0000 7f0a20e00640 1 --2- 172.21.15.26:0/1806369129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a1c162b40 0x7f0a1c0cc670 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0a14007e60 tx=0x7f0a1400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.558+0000 7f0a13e00640 1 -- 172.21.15.26:0/1806369129 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a14014420 con 0x7f0a1c162b40 2024-04-26T21:10:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.558+0000 7f0a13e00640 1 -- 172.21.15.26:0/1806369129 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a14019070 con 0x7f0a1c162b40 2024-04-26T21:10:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.558+0000 7f0a13e00640 1 -- 172.21.15.26:0/1806369129 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a14014820 con 0x7f0a1c162b40 2024-04-26T21:10:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.558+0000 7f0a21800640 1 -- 172.21.15.26:0/1806369129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a1c0ca9e0 con 0x7f0a1c162b40 2024-04-26T21:10:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.558+0000 7f0a21800640 1 -- 172.21.15.26:0/1806369129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a1c0cafa0 con 0x7f0a1c162b40 2024-04-26T21:10:35.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.559+0000 7f0a21800640 1 -- 172.21.15.26:0/1806369129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0a1c06a700 con 0x7f0a1c162b40 2024-04-26T21:10:35.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.560+0000 7f0a13e00640 1 -- 172.21.15.26:0/1806369129 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a140149c0 con 0x7f0a1c162b40 2024-04-26T21:10:35.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.561+0000 7f0a13e00640 1 --2- 172.21.15.26:0/1806369129 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f09fc06c920 0x7f09fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:35.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.561+0000 7f0a13e00640 1 -- 172.21.15.26:0/1806369129 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f0a1408bfe0 con 0x7f0a1c162b40 2024-04-26T21:10:35.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.561+0000 7f0a1be00640 1 --2- 172.21.15.26:0/1806369129 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f09fc06c920 0x7f09fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:35.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.561+0000 7f0a13e00640 1 --2- 172.21.15.26:0/1806369129 >> [v2:172.21.15.86:6808/2582257935,v1:172.21.15.86:6809/2582257935] conn(0x7f09fc0727e0 0x7f09fc074c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:35.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.561+0000 7f0a13e00640 1 -- 172.21.15.26:0/1806369129 --> [v2:172.21.15.86:6808/2582257935,v1:172.21.15.86:6809/2582257935] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a14014e50 con 0x7f09fc0727e0 2024-04-26T21:10:35.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.561+0000 7f0a13e00640 1 -- 172.21.15.26:0/1806369129 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0a1408e5c0 con 0x7f0a1c162b40 2024-04-26T21:10:35.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.562+0000 7f0a1b400640 1 --2- 172.21.15.26:0/1806369129 >> [v2:172.21.15.86:6808/2582257935,v1:172.21.15.86:6809/2582257935] conn(0x7f09fc0727e0 0x7f09fc074c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:35.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.562+0000 7f0a1be00640 1 --2- 172.21.15.26:0/1806369129 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f09fc06c920 0x7f09fc06ede0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c034000 tx=0x7f0a0c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:35.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.563+0000 7f0a1b400640 1 --2- 172.21.15.26:0/1806369129 >> [v2:172.21.15.86:6808/2582257935,v1:172.21.15.86:6809/2582257935] conn(0x7f09fc0727e0 0x7f09fc074c00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:35.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.566+0000 7f0a13e00640 1 -- 172.21.15.26:0/1806369129 <== osd.7 v2:172.21.15.86:6808/2582257935 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a14014e50 con 0x7f09fc0727e0 2024-04-26T21:10:35.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.580+0000 7f0a21800640 1 -- 172.21.15.26:0/1806369129 --> [v2:172.21.15.86:6808/2582257935,v1:172.21.15.86:6809/2582257935] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0a1c08cf70 con 0x7f09fc0727e0 2024-04-26T21:10:35.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.581+0000 7f0a13e00640 1 -- 172.21.15.26:0/1806369129 <== osd.7 v2:172.21.15.86:6808/2582257935 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0a1c08cf70 con 0x7f09fc0727e0 2024-04-26T21:10:35.582 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-04-26T21:10:35.583 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:10:35.583 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-04-26T21:10:35.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.582+0000 7f0a21800640 1 -- 172.21.15.26:0/1806369129 >> [v2:172.21.15.86:6808/2582257935,v1:172.21.15.86:6809/2582257935] conn(0x7f09fc0727e0 msgr2=0x7f09fc074c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:35.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.582+0000 7f0a21800640 1 --2- 172.21.15.26:0/1806369129 >> [v2:172.21.15.86:6808/2582257935,v1:172.21.15.86:6809/2582257935] conn(0x7f09fc0727e0 0x7f09fc074c00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:35.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.583+0000 7f0a21800640 1 -- 172.21.15.26:0/1806369129 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f09fc06c920 msgr2=0x7f09fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:35.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.583+0000 7f0a21800640 1 --2- 172.21.15.26:0/1806369129 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f09fc06c920 0x7f09fc06ede0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c034000 tx=0x7f0a0c02e000 comp rx=0 tx=0).stop 2024-04-26T21:10:35.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.583+0000 7f0a21800640 1 -- 172.21.15.26:0/1806369129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a1c162b40 msgr2=0x7f0a1c0cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:35.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.583+0000 7f0a21800640 1 --2- 172.21.15.26:0/1806369129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a1c162b40 0x7f0a1c0cc670 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0a14007e60 tx=0x7f0a1400e4b0 comp rx=0 tx=0).stop 2024-04-26T21:10:35.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.583+0000 7f0a20e00640 1 -- 172.21.15.26:0/1806369129 reap_dead start 2024-04-26T21:10:35.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.583+0000 7f0a21800640 1 -- 172.21.15.26:0/1806369129 shutdown_connections 2024-04-26T21:10:35.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.583+0000 7f0a21800640 1 -- 172.21.15.26:0/1806369129 >> 172.21.15.26:0/1806369129 conn(0x7f0a1c0d5060 msgr2=0x7f0a1c06dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:35.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.583+0000 7f0a21800640 1 -- 172.21.15.26:0/1806369129 shutdown_connections 2024-04-26T21:10:35.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.583+0000 7f0a21800640 1 -- 172.21.15.26:0/1806369129 wait complete. 2024-04-26T21:10:35.603 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-26T21:10:35.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.815+0000 7f0367200640 1 Processor -- start 2024-04-26T21:10:35.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.815+0000 7f0367200640 1 -- start start 2024-04-26T21:10:35.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.817+0000 7f0367200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0368162ba0 0x7f03680d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.817+0000 7f0367200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03680d2060 0x7f03680d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.817+0000 7f0367200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03680b61c0 con 0x7f03680d1c80 2024-04-26T21:10:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.817+0000 7f0367200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 -- 0x7f03680b5f70 con 0x7f03680d2060 2024-04-26T21:10:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.817+0000 7f0367200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f03680d3830 con 0x7f0368162ba0 2024-04-26T21:10:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.817+0000 7f0366800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03680d2060 0x7f03680d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.818+0000 7f0366800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03680d2060 0x7f03680d3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48652/0 (socket says 172.21.15.26:48652) 2024-04-26T21:10:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.818+0000 7f0366800640 1 -- 172.21.15.26:0/461608482 learned_addr learned my addr 172.21.15.26:0/461608482 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.818+0000 7f0365400640 1 --2- 172.21.15.26:0/461608482 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0368162ba0 0x7f03680d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.818+0000 7f0366800640 1 -- 172.21.15.26:0/461608482 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0368162ba0 msgr2=0x7f03680d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.818+0000 7f0366800640 1 --2- 172.21.15.26:0/461608482 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0368162ba0 0x7f03680d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.818+0000 7f0366800640 1 -- 172.21.15.26:0/461608482 >> v1:172.21.15.86:6789/0 conn(0x7f03680d1c80 legacy=0x7f0368162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.818+0000 7f0366800640 1 -- 172.21.15.26:0/461608482 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03680b64a0 con 0x7f03680d2060 2024-04-26T21:10:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.819+0000 7f0366800640 1 --2- 172.21.15.26:0/461608482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03680d2060 0x7f03680d3cc0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0360009e80 tx=0x7f036002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0ebc77e6b5aa2f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.820+0000 7f0364a00640 1 -- 172.21.15.26:0/461608482 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360036070 con 0x7f03680d2060 2024-04-26T21:10:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.820+0000 7f0364a00640 1 -- 172.21.15.26:0/461608482 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0360031600 con 0x7f03680d2060 2024-04-26T21:10:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.820+0000 7f0364a00640 1 -- 172.21.15.26:0/461608482 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03600319d0 con 0x7f03680d2060 2024-04-26T21:10:35.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.820+0000 7f0367200640 1 -- 172.21.15.26:0/461608482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03680d2060 msgr2=0x7f03680d3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:35.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.820+0000 7f0367200640 1 --2- 172.21.15.26:0/461608482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03680d2060 0x7f03680d3cc0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0360009e80 tx=0x7f036002a660 comp rx=0 tx=0).stop 2024-04-26T21:10:35.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.820+0000 7f0367200640 1 -- 172.21.15.26:0/461608482 shutdown_connections 2024-04-26T21:10:35.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.820+0000 7f0367200640 1 --2- 172.21.15.26:0/461608482 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0368162ba0 0x7f03680d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:35.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.820+0000 7f0367200640 1 --2- 172.21.15.26:0/461608482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03680d2060 0x7f03680d3cc0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:35.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.820+0000 7f0367200640 1 -- 172.21.15.26:0/461608482 >> 172.21.15.26:0/461608482 conn(0x7f03680d5020 msgr2=0x7f03680d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:35.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.821+0000 7f0367200640 1 -- 172.21.15.26:0/461608482 shutdown_connections 2024-04-26T21:10:35.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.821+0000 7f0367200640 1 -- 172.21.15.26:0/461608482 wait complete. 2024-04-26T21:10:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.822+0000 7f0367200640 1 Processor -- start 2024-04-26T21:10:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.822+0000 7f0367200640 1 -- start start 2024-04-26T21:10:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.823+0000 7f0367200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03680d1c80 0x7f036814e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.823+0000 7f0367200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f03680d2060 0x7f03680b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.823+0000 7f0367200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0368162ba0 0x7f03680b8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.823+0000 7f0367200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f03680d5cc0 con 0x7f0368162ba0 2024-04-26T21:10:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.823+0000 7f0367200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f03680d5b40 con 0x7f03680d1c80 2024-04-26T21:10:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.823+0000 7f0367200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f03680c8b70 con 0x7f03680d2060 2024-04-26T21:10:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.824+0000 7f0366800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0368162ba0 0x7f03680b8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.824+0000 7f0366800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0368162ba0 0x7f03680b8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48666/0 (socket says 172.21.15.26:48666) 2024-04-26T21:10:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.824+0000 7f0366800640 1 -- 172.21.15.26:0/506673571 learned_addr learned my addr 172.21.15.26:0/506673571 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:35.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.824+0000 7f0365e00640 1 --2- 172.21.15.26:0/506673571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03680d1c80 0x7f036814e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:35.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.824+0000 7f0365400640 1 --2- 172.21.15.26:0/506673571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f03680d2060 0x7f03680b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:35.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.824+0000 7f0366800640 1 -- 172.21.15.26:0/506673571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f03680d2060 msgr2=0x7f03680b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:35.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.824+0000 7f0366800640 1 --2- 172.21.15.26:0/506673571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f03680d2060 0x7f03680b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:35.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.824+0000 7f0366800640 1 -- 172.21.15.26:0/506673571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03680d1c80 msgr2=0x7f036814e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:35.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.824+0000 7f0366800640 1 --2- 172.21.15.26:0/506673571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03680d1c80 0x7f036814e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:35.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.824+0000 7f0366800640 1 -- 172.21.15.26:0/506673571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03680b4db0 con 0x7f0368162ba0 2024-04-26T21:10:35.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.824+0000 7f0365400640 1 --2- 172.21.15.26:0/506673571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f03680d2060 0x7f03680b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:35.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.825+0000 7f0366800640 1 --2- 172.21.15.26:0/506673571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0368162ba0 0x7f03680b8500 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0360009fa0 tx=0x7f0360004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:35.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.825+0000 7f035e000640 1 -- 172.21.15.26:0/506673571 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360036050 con 0x7f0368162ba0 2024-04-26T21:10:35.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.825+0000 7f035e000640 1 -- 172.21.15.26:0/506673571 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0360042070 con 0x7f0368162ba0 2024-04-26T21:10:35.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.825+0000 7f035e000640 1 -- 172.21.15.26:0/506673571 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360009510 con 0x7f0368162ba0 2024-04-26T21:10:35.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.825+0000 7f0367200640 1 -- 172.21.15.26:0/506673571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03680b5090 con 0x7f0368162ba0 2024-04-26T21:10:35.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.825+0000 7f0367200640 1 -- 172.21.15.26:0/506673571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03680b5650 con 0x7f0368162ba0 2024-04-26T21:10:35.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.826+0000 7f0367200640 1 -- 172.21.15.26:0/506673571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f036806a700 con 0x7f0368162ba0 2024-04-26T21:10:35.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.827+0000 7f035e000640 1 -- 172.21.15.26:0/506673571 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0360007c30 con 0x7f0368162ba0 2024-04-26T21:10:35.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.828+0000 7f035e000640 1 --2- 172.21.15.26:0/506673571 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f034806c8f0 0x7f034806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:35.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.828+0000 7f035e000640 1 -- 172.21.15.26:0/506673571 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f03600ac6e0 con 0x7f0368162ba0 2024-04-26T21:10:35.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.829+0000 7f0365e00640 1 --2- 172.21.15.26:0/506673571 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f034806c8f0 0x7f034806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:35.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.829+0000 7f0365e00640 1 --2- 172.21.15.26:0/506673571 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f034806c8f0 0x7f034806edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f03680cee40 tx=0x7f0358009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:35.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:35.833+0000 7f035e000640 1 -- 172.21.15.26:0/506673571 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0360045360 con 0x7f0368162ba0 2024-04-26T21:10:36.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.035+0000 7f0367200640 1 -- 172.21.15.26:0/506673571 --> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f03680d4180 con 0x7f034806c8f0 2024-04-26T21:10:36.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.036+0000 7f035e000640 1 -- 172.21.15.26:0/506673571 <== mgr.4104 v2:172.21.15.26:6832/3130430358 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f03680d4180 con 0x7f034806c8f0 2024-04-26T21:10:36.037 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 7 to deep-scrub 2024-04-26T21:10:36.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.039+0000 7f0367200640 1 -- 172.21.15.26:0/506673571 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f034806c8f0 msgr2=0x7f034806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:36.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.039+0000 7f0367200640 1 --2- 172.21.15.26:0/506673571 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f034806c8f0 0x7f034806edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f03680cee40 tx=0x7f0358009040 comp rx=0 tx=0).stop 2024-04-26T21:10:36.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.039+0000 7f0367200640 1 -- 172.21.15.26:0/506673571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0368162ba0 msgr2=0x7f03680b8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:36.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.039+0000 7f0367200640 1 --2- 172.21.15.26:0/506673571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0368162ba0 0x7f03680b8500 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0360009fa0 tx=0x7f0360004a40 comp rx=0 tx=0).stop 2024-04-26T21:10:36.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.039+0000 7f0367200640 1 -- 172.21.15.26:0/506673571 shutdown_connections 2024-04-26T21:10:36.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.039+0000 7f0367200640 1 --2- 172.21.15.26:0/506673571 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f034806c8f0 0x7f034806edb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:36.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.039+0000 7f0367200640 1 --2- 172.21.15.26:0/506673571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0368162ba0 0x7f03680b8500 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:36.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.039+0000 7f0367200640 1 --2- 172.21.15.26:0/506673571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f03680d2060 0x7f03680b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:36.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.039+0000 7f0367200640 1 --2- 172.21.15.26:0/506673571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03680d1c80 0x7f036814e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:36.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.039+0000 7f0367200640 1 -- 172.21.15.26:0/506673571 >> 172.21.15.26:0/506673571 conn(0x7f03680d5020 msgr2=0x7f03680d5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:36.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.039+0000 7f0367200640 1 -- 172.21.15.26:0/506673571 shutdown_connections 2024-04-26T21:10:36.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.039+0000 7f0367200640 1 -- 172.21.15.26:0/506673571 wait complete. 2024-04-26T21:10:36.059 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:10:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.279+0000 7f744a000640 1 Processor -- start 2024-04-26T21:10:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.280+0000 7f744a000640 1 -- start start 2024-04-26T21:10:36.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.281+0000 7f744a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f74440d1c80 0x7f74440d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:36.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.281+0000 7f744a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f74440d2060 0x7f7444162490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:36.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.281+0000 7f744a000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7444162e60 con 0x7f74441629d0 2024-04-26T21:10:36.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.281+0000 7f744a000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f74440b5f70 con 0x7f74440d2060 2024-04-26T21:10:36.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.281+0000 7f744a000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f74440d3830 con 0x7f74440d1c80 2024-04-26T21:10:36.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.282+0000 7f7449600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f74441629d0 0x7f74440c8b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33466/0 (socket says 172.21.15.26:33466) 2024-04-26T21:10:36.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.282+0000 7f7449600640 1 -- 172.21.15.26:0/867353532 learned_addr learned my addr 172.21.15.26:0/867353532 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:36.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.282+0000 7f7443e00640 1 --2- 172.21.15.26:0/867353532 >> v2:172.21.15.86:3300/0 conn(0x7f74440d2060 0x7f7444162490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:36.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.282+0000 7f7448c00640 1 --2- 172.21.15.26:0/867353532 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f74440d1c80 0x7f74440d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:36.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.282+0000 7f7443400640 1 -- 172.21.15.26:0/867353532 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 452176177 0 0) 0x7f7444162e60 con 0x7f74441629d0 2024-04-26T21:10:36.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.282+0000 7f7443400640 1 -- 172.21.15.26:0/867353532 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f742c003520 con 0x7f74441629d0 2024-04-26T21:10:36.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.283+0000 7f7443e00640 1 -- 172.21.15.26:0/867353532 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f74440d1c80 msgr2=0x7f74440d3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:36.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.283+0000 7f7443400640 1 -- 172.21.15.26:0/867353532 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1880058523 0 0) 0x7f742c003520 con 0x7f74441629d0 2024-04-26T21:10:36.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.283+0000 7f7443e00640 1 --2- 172.21.15.26:0/867353532 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f74440d1c80 0x7f74440d3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:36.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.283+0000 7f7443e00640 1 -- 172.21.15.26:0/867353532 >> v1:172.21.15.26:6789/0 conn(0x7f74441629d0 legacy=0x7f74440c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:36.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.283+0000 7f7443e00640 1 -- 172.21.15.26:0/867353532 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74440b61c0 con 0x7f74440d2060 2024-04-26T21:10:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.284+0000 7f7443e00640 1 --2- 172.21.15.26:0/867353532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74440d2060 0x7f7444162490 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f74300097a0 tx=0x7f743002a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92c1cb7397234bf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.285+0000 7f7443400640 1 -- 172.21.15.26:0/867353532 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430037070 con 0x7f74440d2060 2024-04-26T21:10:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.285+0000 7f7443400640 1 -- 172.21.15.26:0/867353532 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f743002ab10 con 0x7f74440d2060 2024-04-26T21:10:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.285+0000 7f7443400640 1 -- 172.21.15.26:0/867353532 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c0019f0 con 0x7f74440d2060 2024-04-26T21:10:36.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.285+0000 7f744a000640 1 -- 172.21.15.26:0/867353532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74440d2060 msgr2=0x7f7444162490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:36.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.285+0000 7f744a000640 1 --2- 172.21.15.26:0/867353532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74440d2060 0x7f7444162490 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f74300097a0 tx=0x7f743002a080 comp rx=0 tx=0).stop 2024-04-26T21:10:36.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.286+0000 7f744a000640 1 -- 172.21.15.26:0/867353532 shutdown_connections 2024-04-26T21:10:36.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.286+0000 7f744a000640 1 --2- 172.21.15.26:0/867353532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74440d2060 0x7f7444162490 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:36.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.286+0000 7f744a000640 1 --2- 172.21.15.26:0/867353532 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f74440d1c80 0x7f74440d3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:36.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.286+0000 7f744a000640 1 -- 172.21.15.26:0/867353532 >> 172.21.15.26:0/867353532 conn(0x7f74440d5020 msgr2=0x7f74440d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:36.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.286+0000 7f744a000640 1 -- 172.21.15.26:0/867353532 shutdown_connections 2024-04-26T21:10:36.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.286+0000 7f744a000640 1 -- 172.21.15.26:0/867353532 wait complete. 2024-04-26T21:10:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.287+0000 7f744a000640 1 Processor -- start 2024-04-26T21:10:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.288+0000 7f744a000640 1 -- start start 2024-04-26T21:10:36.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.289+0000 7f744a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74440d1c80 0x7f744414c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:36.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.289+0000 7f744a000640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f74440d2060 0x7f74440d0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:36.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.289+0000 7f744a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74441629d0 0x7f74440d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:36.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.289+0000 7f744a000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7444074790 con 0x7f74440d1c80 2024-04-26T21:10:36.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.289+0000 7f744a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7444074610 con 0x7f74441629d0 2024-04-26T21:10:36.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.289+0000 7f744a000640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f74440d5b40 con 0x7f74440d2060 2024-04-26T21:10:36.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.289+0000 7f7448c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74440d1c80 0x7f744414c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:36.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.289+0000 7f7448c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74440d1c80 0x7f744414c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48668/0 (socket says 172.21.15.26:48668) 2024-04-26T21:10:36.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.289+0000 7f7448c00640 1 -- 172.21.15.26:0/533123004 learned_addr learned my addr 172.21.15.26:0/533123004 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:36.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.289+0000 7f7449600640 1 --2- 172.21.15.26:0/533123004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74441629d0 0x7f74440d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:36.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.289+0000 7f7443e00640 1 --2- 172.21.15.26:0/533123004 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f74440d2060 0x7f74440d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:36.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.289+0000 7f7448c00640 1 -- 172.21.15.26:0/533123004 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f74440d2060 msgr2=0x7f74440d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:36.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.289+0000 7f7448c00640 1 --2- 172.21.15.26:0/533123004 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f74440d2060 0x7f74440d0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:36.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.289+0000 7f7448c00640 1 -- 172.21.15.26:0/533123004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74441629d0 msgr2=0x7f74440d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:36.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.290+0000 7f7448c00640 1 --2- 172.21.15.26:0/533123004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74441629d0 0x7f74440d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:36.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.290+0000 7f7448c00640 1 -- 172.21.15.26:0/533123004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74441f5d90 con 0x7f74440d1c80 2024-04-26T21:10:36.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.290+0000 7f7449600640 1 --2- 172.21.15.26:0/533123004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74441629d0 0x7f74440d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:36.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.290+0000 7f7448c00640 1 --2- 172.21.15.26:0/533123004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74440d1c80 0x7f744414c300 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f743800e8a0 tx=0x7f743800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:36.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.290+0000 7f7440c00640 1 -- 172.21.15.26:0/533123004 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438017070 con 0x7f74440d1c80 2024-04-26T21:10:36.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.290+0000 7f7440c00640 1 -- 172.21.15.26:0/533123004 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f743800c940 con 0x7f74440d1c80 2024-04-26T21:10:36.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.290+0000 7f7440c00640 1 -- 172.21.15.26:0/533123004 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743800cd30 con 0x7f74440d1c80 2024-04-26T21:10:36.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.290+0000 7f744a000640 1 -- 172.21.15.26:0/533123004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74441f87b0 con 0x7f74440d1c80 2024-04-26T21:10:36.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.290+0000 7f744a000640 1 -- 172.21.15.26:0/533123004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74441f8d70 con 0x7f74440d1c80 2024-04-26T21:10:36.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.292+0000 7f744a000640 1 -- 172.21.15.26:0/533123004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f744406a700 con 0x7f74440d1c80 2024-04-26T21:10:36.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.293+0000 7f7440c00640 1 -- 172.21.15.26:0/533123004 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f743801a070 con 0x7f74440d1c80 2024-04-26T21:10:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.294+0000 7f7440c00640 1 --2- 172.21.15.26:0/533123004 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f742406c8f0 0x7f742406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.294+0000 7f7440c00640 1 -- 172.21.15.26:0/533123004 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f743808baa0 con 0x7f74440d1c80 2024-04-26T21:10:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.294+0000 7f7443e00640 1 --2- 172.21.15.26:0/533123004 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f742406c8f0 0x7f742406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.295+0000 7f7443e00640 1 --2- 172.21.15.26:0/533123004 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f742406c8f0 0x7f742406edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7430034000 tx=0x7f743002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:36.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.298+0000 7f7440c00640 1 -- 172.21.15.26:0/533123004 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f743805a1c0 con 0x7f74440d1c80 2024-04-26T21:10:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.504+0000 7f744a000640 1 -- 172.21.15.26:0/533123004 --> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f74441f6070 con 0x7f742406c8f0 2024-04-26T21:10:36.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.506+0000 7f7440c00640 1 -- 172.21.15.26:0/533123004 <== mgr.4104 v2:172.21.15.26:6832/3130430358 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13428 (secure 0 0 0) 0x7f743002ac50 con 0x7f742406c8f0 2024-04-26T21:10:36.507 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:36.507 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-26T21:10:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.509+0000 7f744a000640 1 -- 172.21.15.26:0/533123004 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f742406c8f0 msgr2=0x7f742406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.509+0000 7f744a000640 1 --2- 172.21.15.26:0/533123004 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f742406c8f0 0x7f742406edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7430034000 tx=0x7f743002e000 comp rx=0 tx=0).stop 2024-04-26T21:10:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.509+0000 7f744a000640 1 -- 172.21.15.26:0/533123004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74440d1c80 msgr2=0x7f744414c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.509+0000 7f744a000640 1 --2- 172.21.15.26:0/533123004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74440d1c80 0x7f744414c300 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f743800e8a0 tx=0x7f743800ed60 comp rx=0 tx=0).stop 2024-04-26T21:10:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.509+0000 7f744a000640 1 -- 172.21.15.26:0/533123004 shutdown_connections 2024-04-26T21:10:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.509+0000 7f744a000640 1 --2- 172.21.15.26:0/533123004 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f742406c8f0 0x7f742406edb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.509+0000 7f744a000640 1 --2- 172.21.15.26:0/533123004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74441629d0 0x7f74440d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.509+0000 7f744a000640 1 --2- 172.21.15.26:0/533123004 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f74440d2060 0x7f74440d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.509+0000 7f744a000640 1 --2- 172.21.15.26:0/533123004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74440d1c80 0x7f744414c300 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:36.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.509+0000 7f744a000640 1 -- 172.21.15.26:0/533123004 >> 172.21.15.26:0/533123004 conn(0x7f74440d5020 msgr2=0x7f7444160140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:36.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.509+0000 7f744a000640 1 -- 172.21.15.26:0/533123004 shutdown_connections 2024-04-26T21:10:36.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:36.509+0000 7f744a000640 1 -- 172.21.15.26:0/533123004 wait complete. 2024-04-26T21:10:36.530 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-04-26T21:10:35.327392+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":237752,"kb_used_data":4936,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736968,"statfs":{"total":773094113280,"available":772850655232,"internally_reserved":0,"allocated":5054464,"data_stored":2746560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.277588"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":207,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T21:10:24.056599+0000","last_change":"2024-04-26T21:08:33.504911+0000","last_active":"2024-04-26T21:10:24.056599+0000","last_peered":"2024-04-26T21:10:24.056599+0000","last_clean":"2024-04-26T21:10:24.056599+0000","last_became_active":"2024-04-26T21:08:33.503667+0000","last_became_peered":"2024-04-26T21:08:33.503667+0000","last_unstale":"2024-04-26T21:10:24.056599+0000","last_undegraded":"2024-04-26T21:10:24.056599+0000","last_fullsized":"2024-04-26T21:10:24.056599+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:08:32.013907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:08:32.013907+0000","last_clean_scrub_stamp":"2024-04-26T21:08:32.013907+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:09.596153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1077248,"data_stored":786096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27496,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":483328,"data_stored":195728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27492,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":483328,"data_stored":195728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27492,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":483328,"data_stored":195728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31652,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":483328,"data_stored":195728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31656,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340184,"statfs":{"total":96636764160,"available":96604348416,"internally_reserved":0,"allocated":483328,"data_stored":195728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31652,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":483328,"data_stored":195728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1077248,"data_stored":786096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:10:36.533 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-26T21:08:32.013907+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=8, tm_sec=32, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=10, tm_sec=29, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T21:10:36.534 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-26T21:10:37.127 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:10:37.135 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:10:56.535 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:10:56.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.755+0000 7f3f99a00640 1 Processor -- start 2024-04-26T21:10:56.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.755+0000 7f3f99a00640 1 -- start start 2024-04-26T21:10:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.756+0000 7f3f99a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3f941488a0 0x7f3f940d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.756+0000 7f3f99a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f940738e0 0x7f3f941600e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.757+0000 7f3f99a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f940b5070 con 0x7f3f9406c240 2024-04-26T21:10:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.757+0000 7f3f99a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3f940d1b50 con 0x7f3f941488a0 2024-04-26T21:10:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.757+0000 7f3f99a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3f940d1590 con 0x7f3f940738e0 2024-04-26T21:10:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.757+0000 7f3f99000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3f9406c240 0x7f3f94162d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35216/0 (socket says 172.21.15.26:35216) 2024-04-26T21:10:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.757+0000 7f3f99000640 1 -- 172.21.15.26:0/2031064230 learned_addr learned my addr 172.21.15.26:0/2031064230 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.757+0000 7f3f93400640 1 --2- 172.21.15.26:0/2031064230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f940738e0 0x7f3f941600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.757+0000 7f3f93e00640 1 --2- 172.21.15.26:0/2031064230 >> v2:172.21.15.86:3300/0 conn(0x7f3f941488a0 0x7f3f940d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.758+0000 7f3f92a00640 1 -- 172.21.15.26:0/2031064230 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3403811010 0 0) 0x7f3f940b5070 con 0x7f3f9406c240 2024-04-26T21:10:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.758+0000 7f3f92a00640 1 -- 172.21.15.26:0/2031064230 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f78003520 con 0x7f3f9406c240 2024-04-26T21:10:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.758+0000 7f3f92a00640 1 -- 172.21.15.26:0/2031064230 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4230941792 0 0) 0x7f3f78003520 con 0x7f3f9406c240 2024-04-26T21:10:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.758+0000 7f3f92a00640 1 -- 172.21.15.26:0/2031064230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f940738e0 msgr2=0x7f3f941600e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.758+0000 7f3f92a00640 1 --2- 172.21.15.26:0/2031064230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f940738e0 0x7f3f941600e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.758+0000 7f3f92a00640 1 -- 172.21.15.26:0/2031064230 >> v2:172.21.15.86:3300/0 conn(0x7f3f941488a0 msgr2=0x7f3f940d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.758+0000 7f3f92a00640 1 --2- 172.21.15.26:0/2031064230 >> v2:172.21.15.86:3300/0 conn(0x7f3f941488a0 0x7f3f940d0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.758+0000 7f3f92a00640 1 -- 172.21.15.26:0/2031064230 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f940d1870 con 0x7f3f9406c240 2024-04-26T21:10:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.758+0000 7f3f92a00640 1 -- 172.21.15.26:0/2031064230 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7f3f8c0019f0 con 0x7f3f9406c240 2024-04-26T21:10:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.758+0000 7f3f92a00640 1 -- 172.21.15.26:0/2031064230 >> v1:172.21.15.26:6789/0 conn(0x7f3f9406c240 legacy=0x7f3f94162d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.758+0000 7f3f92a00640 1 --2- 172.21.15.26:0/2031064230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f940738e0 0x7f3f780039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.758+0000 7f3f92a00640 1 --2- 172.21.15.26:0/2031064230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f78003f50 0x7f3f78004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.758+0000 7f3f92a00640 1 --2- 172.21.15.26:0/2031064230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f78004d40 0x7f3f780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.758+0000 7f3f92a00640 1 -- 172.21.15.26:0/2031064230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3f940d1b50 con 0x7f3f78004d40 2024-04-26T21:10:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.758+0000 7f3f92a00640 1 -- 172.21.15.26:0/2031064230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3f940d1590 con 0x7f3f78003f50 2024-04-26T21:10:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.758+0000 7f3f92a00640 1 -- 172.21.15.26:0/2031064230 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3f7800b9c0 con 0x7f3f940738e0 2024-04-26T21:10:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.759+0000 7f3f99000640 1 --2- 172.21.15.26:0/2031064230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f78004d40 0x7f3f780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.759+0000 7f3f93e00640 1 --2- 172.21.15.26:0/2031064230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f940738e0 0x7f3f780039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.759+0000 7f3f93400640 1 --2- 172.21.15.26:0/2031064230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f78003f50 0x7f3f78004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.759+0000 7f3f99000640 1 -- 172.21.15.26:0/2031064230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f940738e0 msgr2=0x7f3f780039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.759+0000 7f3f99000640 1 --2- 172.21.15.26:0/2031064230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f940738e0 0x7f3f780039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.759+0000 7f3f99000640 1 -- 172.21.15.26:0/2031064230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f78003f50 msgr2=0x7f3f78004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.759+0000 7f3f99000640 1 --2- 172.21.15.26:0/2031064230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f78003f50 0x7f3f78004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.759+0000 7f3f99000640 1 -- 172.21.15.26:0/2031064230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f78003520 con 0x7f3f78004d40 2024-04-26T21:10:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.760+0000 7f3f99000640 1 --2- 172.21.15.26:0/2031064230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f78004d40 0x7f3f780051c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00a950 tx=0x7f3f78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d18bebf32c4985c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.761+0000 7f3f92a00640 1 -- 172.21.15.26:0/2031064230 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c00a4c0 con 0x7f3f78004d40 2024-04-26T21:10:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.761+0000 7f3f92a00640 1 -- 172.21.15.26:0/2031064230 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f8c02f070 con 0x7f3f78004d40 2024-04-26T21:10:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.761+0000 7f3f92a00640 1 -- 172.21.15.26:0/2031064230 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c02bd40 con 0x7f3f78004d40 2024-04-26T21:10:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.761+0000 7f3f99a00640 1 -- 172.21.15.26:0/2031064230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f78004d40 msgr2=0x7f3f780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.761+0000 7f3f99a00640 1 --2- 172.21.15.26:0/2031064230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f78004d40 0x7f3f780051c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00a950 tx=0x7f3f78005830 comp rx=0 tx=0).stop 2024-04-26T21:10:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.761+0000 7f3f99000640 1 -- 172.21.15.26:0/2031064230 reap_dead start 2024-04-26T21:10:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.761+0000 7f3f99a00640 1 -- 172.21.15.26:0/2031064230 shutdown_connections 2024-04-26T21:10:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.761+0000 7f3f99a00640 1 -- 172.21.15.26:0/2031064230 >> 172.21.15.26:0/2031064230 conn(0x7f3f9406dd70 msgr2=0x7f3f940d1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.762+0000 7f3f99a00640 1 -- 172.21.15.26:0/2031064230 shutdown_connections 2024-04-26T21:10:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.762+0000 7f3f99a00640 1 -- 172.21.15.26:0/2031064230 wait complete. 2024-04-26T21:10:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.763+0000 7f3f99a00640 1 Processor -- start 2024-04-26T21:10:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.763+0000 7f3f99a00640 1 -- start start 2024-04-26T21:10:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.764+0000 7f3f99a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f9413da10 0x7f3f9415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.764+0000 7f3f99a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f9413ddf0 0x7f3f94145500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.764+0000 7f3f99a00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f9415e8f0 0x7f3f941f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.764+0000 7f3f99a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3f94148c80 con 0x7f3f9413ddf0 2024-04-26T21:10:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.764+0000 7f3f99a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9415fbc0 con 0x7f3f9413da10 2024-04-26T21:10:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.764+0000 7f3f99a00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7f3f94148e00 con 0x7f3f9415e8f0 2024-04-26T21:10:56.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.765+0000 7f3f93400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f9413ddf0 0x7f3f94145500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:56.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.765+0000 7f3f93400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f9413ddf0 0x7f3f94145500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35468/0 (socket says 172.21.15.26:35468) 2024-04-26T21:10:56.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.765+0000 7f3f93400640 1 -- 172.21.15.26:0/3379875638 learned_addr learned my addr 172.21.15.26:0/3379875638 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.765+0000 7f3f93e00640 1 --2- 172.21.15.26:0/3379875638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f9413da10 0x7f3f9415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.765+0000 7f3f99000640 1 --2- 172.21.15.26:0/3379875638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f9415e8f0 0x7f3f941f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.765+0000 7f3f93400640 1 -- 172.21.15.26:0/3379875638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f9415e8f0 msgr2=0x7f3f941f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.765+0000 7f3f93400640 1 --2- 172.21.15.26:0/3379875638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f9415e8f0 0x7f3f941f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.765+0000 7f3f93400640 1 -- 172.21.15.26:0/3379875638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f9413da10 msgr2=0x7f3f9415e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.765+0000 7f3f93400640 1 --2- 172.21.15.26:0/3379875638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f9413da10 0x7f3f9415e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.765+0000 7f3f93400640 1 -- 172.21.15.26:0/3379875638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f94140b70 con 0x7f3f9413ddf0 2024-04-26T21:10:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.765+0000 7f3f99000640 1 --2- 172.21.15.26:0/3379875638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f9415e8f0 0x7f3f941f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:10:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.766+0000 7f3f93400640 1 --2- 172.21.15.26:0/3379875638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f9413ddf0 0x7f3f94145500 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3f84011570 tx=0x7f3f84011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.766+0000 7f3f8be00640 1 -- 172.21.15.26:0/3379875638 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f84015930 con 0x7f3f9413ddf0 2024-04-26T21:10:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.766+0000 7f3f8be00640 1 -- 172.21.15.26:0/3379875638 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f84015ad0 con 0x7f3f9413ddf0 2024-04-26T21:10:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.766+0000 7f3f8be00640 1 -- 172.21.15.26:0/3379875638 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8401d400 con 0x7f3f9413ddf0 2024-04-26T21:10:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.766+0000 7f3f99a00640 1 -- 172.21.15.26:0/3379875638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f94144460 con 0x7f3f9413ddf0 2024-04-26T21:10:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.766+0000 7f3f99a00640 1 -- 172.21.15.26:0/3379875638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f94144a20 con 0x7f3f9413ddf0 2024-04-26T21:10:56.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.767+0000 7f3f99a00640 1 -- 172.21.15.26:0/3379875638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f940693d0 con 0x7f3f9413ddf0 2024-04-26T21:10:56.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.768+0000 7f3f8be00640 1 -- 172.21.15.26:0/3379875638 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f8401d5a0 con 0x7f3f9413ddf0 2024-04-26T21:10:56.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.769+0000 7f3f8be00640 1 --2- 172.21.15.26:0/3379875638 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3f7006c920 0x7f3f7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:56.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.769+0000 7f3f8be00640 1 -- 172.21.15.26:0/3379875638 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f3f8408c110 con 0x7f3f9413ddf0 2024-04-26T21:10:56.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.770+0000 7f3f93e00640 1 --2- 172.21.15.26:0/3379875638 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3f7006c920 0x7f3f7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.770+0000 7f3f93e00640 1 --2- 172.21.15.26:0/3379875638 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3f7006c920 0x7f3f7006ede0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009a90 tx=0x7f3f7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.774+0000 7f3f8be00640 1 -- 172.21.15.26:0/3379875638 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f84019030 con 0x7f3f9413ddf0 2024-04-26T21:10:56.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.974+0000 7f3f99a00640 1 -- 172.21.15.26:0/3379875638 --> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3f940736c0 con 0x7f3f7006c920 2024-04-26T21:10:56.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.977+0000 7f3f8be00640 1 -- 172.21.15.26:0/3379875638 <== mgr.4104 v2:172.21.15.26:6832/3130430358 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13359 (secure 0 0 0) 0x7f3f940736c0 con 0x7f3f7006c920 2024-04-26T21:10:56.978 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:56.978 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-04-26T21:10:56.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.980+0000 7f3f99a00640 1 -- 172.21.15.26:0/3379875638 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3f7006c920 msgr2=0x7f3f7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:56.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.980+0000 7f3f99a00640 1 --2- 172.21.15.26:0/3379875638 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3f7006c920 0x7f3f7006ede0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c009a90 tx=0x7f3f7c008040 comp rx=0 tx=0).stop 2024-04-26T21:10:56.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.980+0000 7f3f99a00640 1 -- 172.21.15.26:0/3379875638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f9413ddf0 msgr2=0x7f3f94145500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:56.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.980+0000 7f3f99a00640 1 --2- 172.21.15.26:0/3379875638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f9413ddf0 0x7f3f94145500 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3f84011570 tx=0x7f3f84011a30 comp rx=0 tx=0).stop 2024-04-26T21:10:56.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.980+0000 7f3f99a00640 1 -- 172.21.15.26:0/3379875638 shutdown_connections 2024-04-26T21:10:56.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.980+0000 7f3f99a00640 1 --2- 172.21.15.26:0/3379875638 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7f3f7006c920 0x7f3f7006ede0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.980+0000 7f3f99a00640 1 --2- 172.21.15.26:0/3379875638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f9415e8f0 0x7f3f941f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.980+0000 7f3f99a00640 1 --2- 172.21.15.26:0/3379875638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f9413ddf0 0x7f3f94145500 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.980+0000 7f3f99a00640 1 --2- 172.21.15.26:0/3379875638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f9413da10 0x7f3f9415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:56.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.980+0000 7f3f99a00640 1 -- 172.21.15.26:0/3379875638 >> 172.21.15.26:0/3379875638 conn(0x7f3f9406dd70 msgr2=0x7f3f94162be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:56.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.980+0000 7f3f99a00640 1 -- 172.21.15.26:0/3379875638 shutdown_connections 2024-04-26T21:10:56.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:56.981+0000 7f3f99a00640 1 -- 172.21.15.26:0/3379875638 wait complete. 2024-04-26T21:10:57.003 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-04-26T21:10:55.332917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":237736,"kb_used_data":4936,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736984,"statfs":{"total":773094113280,"available":772850671616,"internally_reserved":0,"allocated":5054464,"data_stored":2746560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003507"},"pg_stats":[{"pgid":"1.0","version":"41'119","reported_seq":219,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T21:10:36.558248+0000","last_change":"2024-04-26T21:10:36.558401+0000","last_active":"2024-04-26T21:10:36.558248+0000","last_peered":"2024-04-26T21:10:36.558248+0000","last_clean":"2024-04-26T21:10:36.558248+0000","last_became_active":"2024-04-26T21:08:33.503667+0000","last_became_peered":"2024-04-26T21:08:33.503667+0000","last_unstale":"2024-04-26T21:10:36.558248+0000","last_undegraded":"2024-04-26T21:10:36.558248+0000","last_fullsized":"2024-04-26T21:10:36.558248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"41'119","last_scrub_stamp":"2024-04-26T21:10:36.558023+0000","last_deep_scrub":"41'119","last_deep_scrub_stamp":"2024-04-26T21:10:36.558023+0000","last_clean_scrub_stamp":"2024-04-26T21:10:36.558023+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:19:14.982491+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343772,"statfs":{"total":96636764160,"available":96608022528,"internally_reserved":0,"allocated":1077248,"data_stored":786096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27496,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":483328,"data_stored":195728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27492,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":483328,"data_stored":195728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27492,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":483328,"data_stored":195728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31652,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":483328,"data_stored":195728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31656,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340184,"statfs":{"total":96636764160,"available":96604348416,"internally_reserved":0,"allocated":483328,"data_stored":195728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31652,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":483328,"data_stored":195728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1077248,"data_stored":786096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:10:57.004 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-26T21:10:57.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.192+0000 7fe225400640 1 Processor -- start 2024-04-26T21:10:57.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.192+0000 7fe225400640 1 -- start start 2024-04-26T21:10:57.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.193+0000 7fe225400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe22015f290 0x7fe2200d5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:57.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.193+0000 7fe225400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe220163570 0x7fe2200b8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:57.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.193+0000 7fe225400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2200d3870 con 0x7fe2200d5c40 2024-04-26T21:10:57.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.193+0000 7fe225400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe2200d3b50 con 0x7fe220163570 2024-04-26T21:10:57.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.193+0000 7fe225400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe2200d3e30 con 0x7fe22015f290 2024-04-26T21:10:57.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.194+0000 7fe21fe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe22015f290 0x7fe2200d5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:57.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.194+0000 7fe224a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe220163570 0x7fe2200b8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:57.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.194+0000 7fe21fe00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe22015f290 0x7fe2200d5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.26:60950/0 (socket says 172.21.15.26:60950) 2024-04-26T21:10:57.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.194+0000 7fe21fe00640 1 -- 172.21.15.26:0/238331267 learned_addr learned my addr 172.21.15.26:0/238331267 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:57.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.195+0000 7fe224a00640 1 -- 172.21.15.26:0/238331267 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe22015f290 msgr2=0x7fe2200d5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:57.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.195+0000 7fe224a00640 1 --2- 172.21.15.26:0/238331267 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe22015f290 0x7fe2200d5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:57.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.195+0000 7fe224a00640 1 -- 172.21.15.26:0/238331267 >> v1:172.21.15.86:6789/0 conn(0x7fe2200d5c40 legacy=0x7fe220162e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:57.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.195+0000 7fe224a00640 1 -- 172.21.15.26:0/238331267 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2200b7e40 con 0x7fe220163570 2024-04-26T21:10:57.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.196+0000 7fe224a00640 1 --2- 172.21.15.26:0/238331267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe220163570 0x7fe2200b8640 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe218009d90 tx=0x7fe21802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba2865f8fbdeb11e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:57.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.197+0000 7fe21f400640 1 -- 172.21.15.26:0/238331267 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218007740 con 0x7fe220163570 2024-04-26T21:10:57.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.197+0000 7fe21f400640 1 -- 172.21.15.26:0/238331267 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2180078e0 con 0x7fe220163570 2024-04-26T21:10:57.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.197+0000 7fe21f400640 1 -- 172.21.15.26:0/238331267 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218007cb0 con 0x7fe220163570 2024-04-26T21:10:57.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.198+0000 7fe225400640 1 -- 172.21.15.26:0/238331267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe220163570 msgr2=0x7fe2200b8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:57.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.198+0000 7fe225400640 1 --2- 172.21.15.26:0/238331267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe220163570 0x7fe2200b8640 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe218009d90 tx=0x7fe21802a660 comp rx=0 tx=0).stop 2024-04-26T21:10:57.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.198+0000 7fe225400640 1 -- 172.21.15.26:0/238331267 shutdown_connections 2024-04-26T21:10:57.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.198+0000 7fe225400640 1 --2- 172.21.15.26:0/238331267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe220163570 0x7fe2200b8640 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:57.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.198+0000 7fe225400640 1 --2- 172.21.15.26:0/238331267 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe22015f290 0x7fe2200d5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:57.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.198+0000 7fe225400640 1 -- 172.21.15.26:0/238331267 >> 172.21.15.26:0/238331267 conn(0x7fe22009e380 msgr2=0x7fe2200d4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:57.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.198+0000 7fe225400640 1 -- 172.21.15.26:0/238331267 shutdown_connections 2024-04-26T21:10:57.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.198+0000 7fe225400640 1 -- 172.21.15.26:0/238331267 wait complete. 2024-04-26T21:10:57.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.200+0000 7fe225400640 1 Processor -- start 2024-04-26T21:10:57.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.200+0000 7fe225400640 1 -- start start 2024-04-26T21:10:57.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.201+0000 7fe225400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2200d5c40 0x7fe2200cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:57.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.201+0000 7fe225400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe22015f290 0x7fe22015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:57.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.201+0000 7fe225400640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe220163570 0x7fe2200d1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:57.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.201+0000 7fe225400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe22015f670 con 0x7fe2200d5c40 2024-04-26T21:10:57.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.201+0000 7fe225400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2200d3540 con 0x7fe22015f290 2024-04-26T21:10:57.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.201+0000 7fe225400640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fe2200b8b80 con 0x7fe220163570 2024-04-26T21:10:57.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.201+0000 7fe21fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2200d5c40 0x7fe2200cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:57.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.201+0000 7fe21fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2200d5c40 0x7fe2200cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35480/0 (socket says 172.21.15.26:35480) 2024-04-26T21:10:57.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.201+0000 7fe21fe00640 1 -- 172.21.15.26:0/2603203262 learned_addr learned my addr 172.21.15.26:0/2603203262 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:57.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.202+0000 7fe224a00640 1 --2- 172.21.15.26:0/2603203262 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe220163570 0x7fe2200d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:57.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.202+0000 7fe217400640 1 --2- 172.21.15.26:0/2603203262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe22015f290 0x7fe22015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:57.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.202+0000 7fe21fe00640 1 -- 172.21.15.26:0/2603203262 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe220163570 msgr2=0x7fe2200d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:57.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.202+0000 7fe21fe00640 1 --2- 172.21.15.26:0/2603203262 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe220163570 0x7fe2200d1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:57.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.202+0000 7fe21fe00640 1 -- 172.21.15.26:0/2603203262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe22015f290 msgr2=0x7fe22015b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:57.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.202+0000 7fe21fe00640 1 --2- 172.21.15.26:0/2603203262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe22015f290 0x7fe22015b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:57.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.202+0000 7fe21fe00640 1 -- 172.21.15.26:0/2603203262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2200c8a90 con 0x7fe2200d5c40 2024-04-26T21:10:57.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.202+0000 7fe21fe00640 1 --2- 172.21.15.26:0/2603203262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2200d5c40 0x7fe2200cba60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe210009fd0 tx=0x7fe21000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:57.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.202+0000 7fe21cc00640 1 -- 172.21.15.26:0/2603203262 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210017070 con 0x7fe2200d5c40 2024-04-26T21:10:57.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.202+0000 7fe21cc00640 1 -- 172.21.15.26:0/2603203262 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe21000c940 con 0x7fe2200d5c40 2024-04-26T21:10:57.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.202+0000 7fe21cc00640 1 -- 172.21.15.26:0/2603203262 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21000cd10 con 0x7fe2200d5c40 2024-04-26T21:10:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.202+0000 7fe225400640 1 -- 172.21.15.26:0/2603203262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe220161850 con 0x7fe2200d5c40 2024-04-26T21:10:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.203+0000 7fe225400640 1 -- 172.21.15.26:0/2603203262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2200cc440 con 0x7fe2200d5c40 2024-04-26T21:10:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.204+0000 7fe225400640 1 -- 172.21.15.26:0/2603203262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe22006a700 con 0x7fe2200d5c40 2024-04-26T21:10:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.205+0000 7fe21cc00640 1 -- 172.21.15.26:0/2603203262 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe21001a070 con 0x7fe2200d5c40 2024-04-26T21:10:57.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.206+0000 7fe21cc00640 1 --2- 172.21.15.26:0/2603203262 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe1fc06c8f0 0x7fe1fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:57.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.206+0000 7fe21cc00640 1 -- 172.21.15.26:0/2603203262 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7fe21008b7a0 con 0x7fe2200d5c40 2024-04-26T21:10:57.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.206+0000 7fe217400640 1 --2- 172.21.15.26:0/2603203262 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe1fc06c8f0 0x7fe1fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:57.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.207+0000 7fe217400640 1 --2- 172.21.15.26:0/2603203262 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe1fc06c8f0 0x7fe1fc06edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe208007fd0 tx=0x7fe208009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:57.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.211+0000 7fe21cc00640 1 -- 172.21.15.26:0/2603203262 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe2200c8a90 con 0x7fe2200d5c40 2024-04-26T21:10:57.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.418+0000 7fe225400640 1 -- 172.21.15.26:0/2603203262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fe2200b76e0 con 0x7fe2200d5c40 2024-04-26T21:10:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.425+0000 7fe21cc00640 1 -- 172.21.15.26:0/2603203262 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v15)=0 v15) ==== 123+0+0 (secure 0 0 0) 0x7fe210059ec0 con 0x7fe2200d5c40 2024-04-26T21:10:57.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.427+0000 7fe21cc00640 1 -- 172.21.15.26:0/2603203262 <== mon.0 v2:172.21.15.26:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe2100597c0 con 0x7fe2200d5c40 2024-04-26T21:10:57.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.428+0000 7fe225400640 1 -- 172.21.15.26:0/2603203262 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe1fc06c8f0 msgr2=0x7fe1fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:57.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.428+0000 7fe225400640 1 --2- 172.21.15.26:0/2603203262 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe1fc06c8f0 0x7fe1fc06edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe208007fd0 tx=0x7fe208009040 comp rx=0 tx=0).stop 2024-04-26T21:10:57.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.429+0000 7fe225400640 1 -- 172.21.15.26:0/2603203262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2200d5c40 msgr2=0x7fe2200cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:57.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.429+0000 7fe225400640 1 --2- 172.21.15.26:0/2603203262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2200d5c40 0x7fe2200cba60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe210009fd0 tx=0x7fe21000ed40 comp rx=0 tx=0).stop 2024-04-26T21:10:57.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.429+0000 7fe225400640 1 -- 172.21.15.26:0/2603203262 shutdown_connections 2024-04-26T21:10:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.429+0000 7fe225400640 1 --2- 172.21.15.26:0/2603203262 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fe1fc06c8f0 0x7fe1fc06edb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.429+0000 7fe225400640 1 --2- 172.21.15.26:0/2603203262 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe220163570 0x7fe2200d1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.429+0000 7fe225400640 1 --2- 172.21.15.26:0/2603203262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe22015f290 0x7fe22015b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.429+0000 7fe225400640 1 --2- 172.21.15.26:0/2603203262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2200d5c40 0x7fe2200cba60 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.429+0000 7fe225400640 1 -- 172.21.15.26:0/2603203262 >> 172.21.15.26:0/2603203262 conn(0x7fe22009e380 msgr2=0x7fe2200ba8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.429+0000 7fe225400640 1 -- 172.21.15.26:0/2603203262 shutdown_connections 2024-04-26T21:10:57.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.429+0000 7fe225400640 1 -- 172.21.15.26:0/2603203262 wait complete. 2024-04-26T21:10:57.450 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:10:57.450 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:10:57.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.632+0000 7fa42d200640 1 Processor -- start 2024-04-26T21:10:57.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.632+0000 7fa42d200640 1 -- start start 2024-04-26T21:10:57.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.633+0000 7fa42d200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4280b5660 0x7fa4280b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:57.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.633+0000 7fa42d200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa42815fe90 0x7fa4281602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:57.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.633+0000 7fa42d200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4280b5f80 con 0x7fa4280731c0 2024-04-26T21:10:57.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.633+0000 7fa42d200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa42815d170 con 0x7fa42815fe90 2024-04-26T21:10:57.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.633+0000 7fa42d200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa42815c580 con 0x7fa4280b5660 2024-04-26T21:10:57.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.634+0000 7fa426800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa4280731c0 0x7fa428073630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35228/0 (socket says 172.21.15.26:35228) 2024-04-26T21:10:57.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.634+0000 7fa426800640 1 -- 172.21.15.26:0/2060075799 learned_addr learned my addr 172.21.15.26:0/2060075799 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-04-26T21:10:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.634+0000 7fa425e00640 1 --2- 172.21.15.26:0/2060075799 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4280b5660 0x7fa4280b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.634+0000 7fa425400640 1 --2- 172.21.15.26:0/2060075799 >> v2:172.21.15.86:3300/0 conn(0x7fa42815fe90 0x7fa4281602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.634+0000 7fa424a00640 1 -- 172.21.15.26:0/2060075799 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3196434466 0 0) 0x7fa4280b5f80 con 0x7fa4280731c0 2024-04-26T21:10:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.634+0000 7fa424a00640 1 -- 172.21.15.26:0/2060075799 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa40c005730 con 0x7fa4280731c0 2024-04-26T21:10:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.635+0000 7fa424a00640 1 -- 172.21.15.26:0/2060075799 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4036557869 0 0) 0x7fa40c005730 con 0x7fa4280731c0 2024-04-26T21:10:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.635+0000 7fa424a00640 1 -- 172.21.15.26:0/2060075799 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4280b5660 msgr2=0x7fa4280b5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.635+0000 7fa424a00640 1 --2- 172.21.15.26:0/2060075799 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4280b5660 0x7fa4280b5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.635+0000 7fa424a00640 1 -- 172.21.15.26:0/2060075799 >> v2:172.21.15.86:3300/0 conn(0x7fa42815fe90 msgr2=0x7fa4281602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.635+0000 7fa424a00640 1 --2- 172.21.15.26:0/2060075799 >> v2:172.21.15.86:3300/0 conn(0x7fa42815fe90 0x7fa4281602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.635+0000 7fa424a00640 1 -- 172.21.15.26:0/2060075799 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42815c840 con 0x7fa4280731c0 2024-04-26T21:10:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.635+0000 7fa424a00640 1 -- 172.21.15.26:0/2060075799 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3770851633 0 0) 0x7fa420001a10 con 0x7fa4280731c0 2024-04-26T21:10:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.635+0000 7fa424a00640 1 -- 172.21.15.26:0/2060075799 >> v1:172.21.15.26:6789/0 conn(0x7fa4280731c0 legacy=0x7fa428073630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.635+0000 7fa424a00640 1 --2- 172.21.15.26:0/2060075799 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4280b5660 0x7fa40c005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.635+0000 7fa424a00640 1 --2- 172.21.15.26:0/2060075799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa40c006140 0x7fa40c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.635+0000 7fa424a00640 1 --2- 172.21.15.26:0/2060075799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa40c006f50 0x7fa40c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.635+0000 7fa424a00640 1 -- 172.21.15.26:0/2060075799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa42815d170 con 0x7fa40c006f50 2024-04-26T21:10:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.635+0000 7fa424a00640 1 -- 172.21.15.26:0/2060075799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa42815c580 con 0x7fa40c006140 2024-04-26T21:10:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.635+0000 7fa424a00640 1 -- 172.21.15.26:0/2060075799 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa40c00dbd0 con 0x7fa4280b5660 2024-04-26T21:10:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.635+0000 7fa426800640 1 --2- 172.21.15.26:0/2060075799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa40c006f50 0x7fa40c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.635+0000 7fa425e00640 1 --2- 172.21.15.26:0/2060075799 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4280b5660 0x7fa40c005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.635+0000 7fa425400640 1 --2- 172.21.15.26:0/2060075799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa40c006140 0x7fa40c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.636+0000 7fa426800640 1 -- 172.21.15.26:0/2060075799 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4280b5660 msgr2=0x7fa40c005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.636+0000 7fa426800640 1 --2- 172.21.15.26:0/2060075799 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4280b5660 0x7fa40c005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.636+0000 7fa426800640 1 -- 172.21.15.26:0/2060075799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa40c006140 msgr2=0x7fa40c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:10:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.636+0000 7fa426800640 1 --2- 172.21.15.26:0/2060075799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa40c006140 0x7fa40c006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.636+0000 7fa426800640 1 -- 172.21.15.26:0/2060075799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c005730 con 0x7fa40c006f50 2024-04-26T21:10:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.636+0000 7fa425400640 1 --2- 172.21.15.26:0/2060075799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa40c006140 0x7fa40c006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:10:57.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.638+0000 7fa426800640 1 --2- 172.21.15.26:0/2060075799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa40c006f50 0x7fa40c0073d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa42000a880 tx=0x7fa40c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00eca9355f37f74 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:57.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.638+0000 7fa424a00640 1 -- 172.21.15.26:0/2060075799 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42000a4d0 con 0x7fa40c006f50 2024-04-26T21:10:57.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.638+0000 7fa424a00640 1 -- 172.21.15.26:0/2060075799 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa42002f070 con 0x7fa40c006f50 2024-04-26T21:10:57.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.638+0000 7fa424a00640 1 -- 172.21.15.26:0/2060075799 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42002bd30 con 0x7fa40c006f50 2024-04-26T21:10:57.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.639+0000 7fa42d200640 1 -- 172.21.15.26:0/2060075799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa40c006f50 msgr2=0x7fa40c0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:57.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.639+0000 7fa42d200640 1 --2- 172.21.15.26:0/2060075799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa40c006f50 0x7fa40c0073d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa42000a880 tx=0x7fa40c007a40 comp rx=0 tx=0).stop 2024-04-26T21:10:57.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.639+0000 7fa426800640 1 -- 172.21.15.26:0/2060075799 reap_dead start 2024-04-26T21:10:57.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.639+0000 7fa42d200640 1 -- 172.21.15.26:0/2060075799 shutdown_connections 2024-04-26T21:10:57.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.639+0000 7fa42d200640 1 -- 172.21.15.26:0/2060075799 >> 172.21.15.26:0/2060075799 conn(0x7fa4280b3be0 msgr2=0x7fa4280b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:57.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.639+0000 7fa42d200640 1 -- 172.21.15.26:0/2060075799 shutdown_connections 2024-04-26T21:10:57.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.639+0000 7fa42d200640 1 -- 172.21.15.26:0/2060075799 wait complete. 2024-04-26T21:10:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.641+0000 7fa42d200640 1 Processor -- start 2024-04-26T21:10:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.641+0000 7fa42d200640 1 -- start start 2024-04-26T21:10:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.642+0000 7fa42d200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa42815b600 0x7fa428070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.642+0000 7fa42d200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa428070b70 0x7fa4281f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.642+0000 7fa42d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4281f8180 0x7fa4281f8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.642+0000 7fa42d200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa428145d10 con 0x7fa42815b600 2024-04-26T21:10:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.642+0000 7fa42d200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa42815ce00 con 0x7fa4281f8180 2024-04-26T21:10:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.642+0000 7fa42d200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 -- 0x7fa428145e90 con 0x7fa428070b70 2024-04-26T21:10:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.642+0000 7fa425e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa42815b600 0x7fa428070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.642+0000 7fa425e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa42815b600 0x7fa428070630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35500/0 (socket says 172.21.15.26:35500) 2024-04-26T21:10:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.642+0000 7fa425e00640 1 -- 172.21.15.26:0/2777839450 learned_addr learned my addr 172.21.15.26:0/2777839450 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-04-26T21:10:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.643+0000 7fa426800640 1 --2- 172.21.15.26:0/2777839450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4281f8180 0x7fa4281f8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.643+0000 7fa425400640 1 --2- 172.21.15.26:0/2777839450 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa428070b70 0x7fa4281f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.643+0000 7fa425e00640 1 -- 172.21.15.26:0/2777839450 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa428070b70 msgr2=0x7fa4281f7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.643+0000 7fa425e00640 1 --2- 172.21.15.26:0/2777839450 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa428070b70 0x7fa4281f7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.643+0000 7fa425e00640 1 -- 172.21.15.26:0/2777839450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4281f8180 msgr2=0x7fa4281f8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.643+0000 7fa425e00640 1 --2- 172.21.15.26:0/2777839450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4281f8180 0x7fa4281f8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.643+0000 7fa425e00640 1 -- 172.21.15.26:0/2777839450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa428136a60 con 0x7fa42815b600 2024-04-26T21:10:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.643+0000 7fa425e00640 1 --2- 172.21.15.26:0/2777839450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa42815b600 0x7fa428070630 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa4180119d0 tx=0x7fa418011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.643+0000 7fa41e000640 1 -- 172.21.15.26:0/2777839450 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41800fb50 con 0x7fa42815b600 2024-04-26T21:10:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.643+0000 7fa41e000640 1 -- 172.21.15.26:0/2777839450 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa41800fcf0 con 0x7fa42815b600 2024-04-26T21:10:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.643+0000 7fa41e000640 1 -- 172.21.15.26:0/2777839450 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41800fea0 con 0x7fa42815b600 2024-04-26T21:10:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.643+0000 7fa42d200640 1 -- 172.21.15.26:0/2777839450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa428136d40 con 0x7fa42815b600 2024-04-26T21:10:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.643+0000 7fa42d200640 1 -- 172.21.15.26:0/2777839450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa428137300 con 0x7fa42815b600 2024-04-26T21:10:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.645+0000 7fa42d200640 1 -- 172.21.15.26:0/2777839450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4280693d0 con 0x7fa42815b600 2024-04-26T21:10:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.646+0000 7fa41e000640 1 -- 172.21.15.26:0/2777839450 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa418012040 con 0x7fa42815b600 2024-04-26T21:10:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.646+0000 7fa41e000640 1 --2- 172.21.15.26:0/2777839450 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa40806c8f0 0x7fa40806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:10:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.647+0000 7fa41e000640 1 -- 172.21.15.26:0/2777839450 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7fa41808bfa0 con 0x7fa42815b600 2024-04-26T21:10:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.647+0000 7fa425400640 1 --2- 172.21.15.26:0/2777839450 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa40806c8f0 0x7fa40806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:10:57.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.648+0000 7fa425400640 1 --2- 172.21.15.26:0/2777839450 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa40806c8f0 0x7fa40806edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa4280ce610 tx=0x7fa414008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:10:57.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.651+0000 7fa41e000640 1 -- 172.21.15.26:0/2777839450 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa418019030 con 0x7fa42815b600 2024-04-26T21:10:57.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.888+0000 7fa42d200640 1 -- 172.21.15.26:0/2777839450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4280cfb30 con 0x7fa42815b600 2024-04-26T21:10:57.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.889+0000 7fa41e000640 1 -- 172.21.15.26:0/2777839450 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3866 (secure 0 0 0) 0x7fa41805a6d0 con 0x7fa42815b600 2024-04-26T21:10:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-04-26T21:10:57.891 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2349160700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2349160700},{"type":"v1","addr":"172.21.15.26:6837","nonce":2349160700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4598,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6834/2824191762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":2824191762},{"type":"v1","addr":"172.21.15.86:6834","nonce":2824191762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4601,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/263746310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6833","nonce":263746310},{"type":"v1","addr":"172.21.15.86:6835","nonce":263746310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5589,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6836/1814160684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6835","nonce":1814160684},{"type":"v1","addr":"172.21.15.26:6836","nonce":1814160684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-04-26T21:10:57.891 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 20 2024-04-26T21:10:57.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.892+0000 7fa42d200640 1 -- 172.21.15.26:0/2777839450 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa40806c8f0 msgr2=0x7fa40806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:57.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.892+0000 7fa42d200640 1 --2- 172.21.15.26:0/2777839450 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa40806c8f0 0x7fa40806edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa4280ce610 tx=0x7fa414008040 comp rx=0 tx=0).stop 2024-04-26T21:10:57.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.892+0000 7fa42d200640 1 -- 172.21.15.26:0/2777839450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa42815b600 msgr2=0x7fa428070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:10:57.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.892+0000 7fa42d200640 1 --2- 172.21.15.26:0/2777839450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa42815b600 0x7fa428070630 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa4180119d0 tx=0x7fa418011e90 comp rx=0 tx=0).stop 2024-04-26T21:10:57.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.892+0000 7fa42d200640 1 -- 172.21.15.26:0/2777839450 shutdown_connections 2024-04-26T21:10:57.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.892+0000 7fa42d200640 1 --2- 172.21.15.26:0/2777839450 >> [v2:172.21.15.26:6832/3130430358,v1:172.21.15.26:6833/3130430358] conn(0x7fa40806c8f0 0x7fa40806edb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:57.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.892+0000 7fa42d200640 1 --2- 172.21.15.26:0/2777839450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4281f8180 0x7fa4281f8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:57.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.892+0000 7fa42d200640 1 --2- 172.21.15.26:0/2777839450 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa428070b70 0x7fa4281f7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:57.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.892+0000 7fa42d200640 1 --2- 172.21.15.26:0/2777839450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa42815b600 0x7fa428070630 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:10:57.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.892+0000 7fa42d200640 1 -- 172.21.15.26:0/2777839450 >> 172.21.15.26:0/2777839450 conn(0x7fa4280b3be0 msgr2=0x7fa4280b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:10:57.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.893+0000 7fa42d200640 1 -- 172.21.15.26:0/2777839450 shutdown_connections 2024-04-26T21:10:57.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-04-26T21:10:57.893+0000 7fa42d200640 1 -- 172.21.15.26:0/2777839450 wait complete. 2024-04-26T21:10:57.911 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-26T21:10:57.911 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-26T21:10:57.911 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:10:57.911 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:10:57.935 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:10:57.936 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:10:57.936 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:10:58.004 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:10:58.004 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:10:58.004 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:10:58.031 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:10:58.032 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:10:58.032 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:10:58.092 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:10:58.092 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-26T21:10:58.092 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-26T21:10:58.092 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:10:58.139 INFO:tasks.ceph.osd.0:Stopped 2024-04-26T21:10:58.139 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-26T21:10:58.139 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:10:58.197 INFO:tasks.ceph.osd.1:Stopped 2024-04-26T21:10:58.198 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-26T21:10:58.198 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:10:58.251 INFO:tasks.ceph.osd.2:Stopped 2024-04-26T21:10:58.252 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-26T21:10:58.252 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:10:58.317 INFO:tasks.ceph.osd.3:Stopped 2024-04-26T21:10:58.317 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-26T21:10:58.317 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:10:58.380 INFO:tasks.ceph.osd.4:Stopped 2024-04-26T21:10:58.380 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-26T21:10:58.380 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:10:58.477 INFO:tasks.ceph.osd.5:Stopped 2024-04-26T21:10:58.477 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-26T21:10:58.477 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:10:58.561 INFO:tasks.ceph.osd.6:Stopped 2024-04-26T21:10:58.561 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-26T21:10:58.561 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:10:58.624 INFO:tasks.ceph.osd.7:Stopped 2024-04-26T21:10:58.624 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-26T21:10:58.624 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-26T21:10:58.624 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:10:58.693 INFO:tasks.ceph.mgr.y:Stopped 2024-04-26T21:10:58.693 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-26T21:10:58.693 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:10:58.793 INFO:tasks.ceph.mgr.x:Stopped 2024-04-26T21:10:58.793 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-26T21:10:58.793 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-26T21:10:58.793 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:10:58.831 INFO:tasks.ceph.mon.a:Stopped 2024-04-26T21:10:58.831 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-26T21:10:58.832 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:10:58.859 INFO:tasks.ceph.mon.b:Stopped 2024-04-26T21:10:58.859 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-26T21:10:58.859 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:10:58.946 INFO:tasks.ceph.mon.c:Stopped 2024-04-26T21:10:58.946 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-26T21:10:58.946 DEBUG:teuthology.orchestra.run.smithi026:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-26T21:10:59.055 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi026.front.sepia.ceph.com 2024-04-26T21:10:59.055 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-26T21:10:59.297 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi026.front.sepia.ceph.com 2024-04-26T21:10:59.297 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-26T21:10:59.444 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi026.front.sepia.ceph.com 2024-04-26T21:10:59.444 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-26T21:10:59.581 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi026.front.sepia.ceph.com 2024-04-26T21:10:59.581 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-26T21:10:59.751 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-26T21:10:59.752 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-26T21:11:00.002 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-26T21:11:00.002 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-26T21:11:00.152 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-26T21:11:00.153 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-26T21:11:00.284 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-26T21:11:00.284 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-26T21:11:00.455 INFO:tasks.ceph:Archiving mon data... 2024-04-26T21:11:00.456 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675194/data/mon.a.tgz 2024-04-26T21:11:00.457 DEBUG:teuthology.orchestra.run.smithi026:> mktemp 2024-04-26T21:11:00.516 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/tmp.toCo1s2IAz 2024-04-26T21:11:00.516 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.toCo1s2IAz 2024-04-26T21:11:00.681 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0666 /tmp/tmp.toCo1s2IAz 2024-04-26T21:11:00.830 DEBUG:teuthology.orchestra.remote:smithi026:/tmp/tmp.toCo1s2IAz is 300KB 2024-04-26T21:11:00.945 DEBUG:teuthology.orchestra.run.smithi026:> rm -fr /tmp/tmp.toCo1s2IAz 2024-04-26T21:11:01.002 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675194/data/mon.b.tgz 2024-04-26T21:11:01.002 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2024-04-26T21:11:01.060 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.VfAYF8UPkm 2024-04-26T21:11:01.060 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.VfAYF8UPkm 2024-04-26T21:11:01.246 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.VfAYF8UPkm 2024-04-26T21:11:01.390 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.VfAYF8UPkm is 401KB 2024-04-26T21:11:01.497 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.VfAYF8UPkm 2024-04-26T21:11:01.554 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675194/data/mon.c.tgz 2024-04-26T21:11:01.554 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2024-04-26T21:11:01.608 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.AdUsXYMaQf 2024-04-26T21:11:01.608 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.AdUsXYMaQf 2024-04-26T21:11:01.775 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.AdUsXYMaQf 2024-04-26T21:11:01.913 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.AdUsXYMaQf is 402KB 2024-04-26T21:11:02.021 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.AdUsXYMaQf 2024-04-26T21:11:02.077 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-26T21:11:02.078 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T21:11:02.086 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T21:11:02.231 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T21:11:02.287 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T21:11:02.288 INFO:tasks.ceph:Archiving crash dumps... 2024-04-26T21:11:02.289 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675194/remote/smithi026/crash 2024-04-26T21:11:02.290 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T21:11:02.381 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675194/remote/smithi086/crash 2024-04-26T21:11:02.382 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T21:11:02.462 INFO:tasks.ceph:Shutting down logrotate 2024-04-26T21:11:02.462 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:11:02.470 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:11:02.590 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T21:11:02.684 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T21:11:02.772 INFO:tasks.ceph:Compressing logs... 2024-04-26T21:11:02.772 DEBUG:teuthology.orchestra.run.smithi026:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T21:11:02.780 DEBUG:teuthology.orchestra.run.smithi086:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T21:11:02.858 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33478.log 2024-04-26T21:11:02.859 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T21:11:02.859 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43208.log 2024-04-26T21:11:02.859 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33478.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T21:11:02.860 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44623.log 2024-04-26T21:11:02.861 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33478.log.gz 2024-04-26T21:11:02.861 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39250.log 2024-04-26T21:11:02.861 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42578.log 2024-04-26T21:11:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44623.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43208.log.gz 2024-04-26T21:11:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46919.log 2024-04-26T21:11:02.863 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44623.log.gz 2024-04-26T21:11:02.863 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33520.log 2024-04-26T21:11:02.863 INFO:teuthology.orchestra.run.smithi086.stderr: 91.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T21:11:02.863 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.1.39250.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.39250.log.gz 2024-04-26T21:11:02.864 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-26T21:11:02.864 INFO:teuthology.orchestra.run.smithi086.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T21:11:02.864 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43292.log 2024-04-26T21:11:02.864 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42578.log.gz 2024-04-26T21:11:02.866 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33520.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-26T21:11:02.866 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33520.log.gz 2024-04-26T21:11:02.866 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-26T21:11:02.866 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47433.log 2024-04-26T21:11:02.866 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46919.log.gz 2024-04-26T21:11:02.866 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43292.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43292.log.gz 2024-04-26T21:11:02.867 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46656.log 2024-04-26T21:11:02.868 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44451.log 2024-04-26T21:11:02.868 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47433.log.gz 2024-04-26T21:11:02.868 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45318.log 2024-04-26T21:11:02.869 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.39683.log 2024-04-26T21:11:02.869 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-26T21:11:02.869 INFO:teuthology.orchestra.run.smithi086.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-26T21:11:02.870 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.3.39683.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.39683.log.gz 2024-04-26T21:11:02.870 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-26T21:11:02.871 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-26T21:11:02.872 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46656.log.gz 2024-04-26T21:11:02.872 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44451.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44451.log.gz 2024-04-26T21:11:02.872 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47307.log 2024-04-26T21:11:02.872 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41159.log 2024-04-26T21:11:02.873 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45318.log.gz 2024-04-26T21:11:02.873 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43730.log 2024-04-26T21:11:02.873 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47307.log.gz 2024-04-26T21:11:02.874 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-26T21:11:02.874 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33729.log 2024-04-26T21:11:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41159.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41159.log.gz 2024-04-26T21:11:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42248.log 2024-04-26T21:11:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44270.log 2024-04-26T21:11:02.875 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43730.log.gz 2024-04-26T21:11:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38533.log 2024-04-26T21:11:02.875 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42248.log.gz 2024-04-26T21:11:02.877 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44270.log: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.44270.log.gz 2024-04-26T21:11:02.877 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45896.log 2024-04-26T21:11:02.877 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33649.log 2024-04-26T21:11:02.877 INFO:teuthology.orchestra.run.smithi086.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-26T21:11:02.877 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33729.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33729.log.gz 2024-04-26T21:11:02.879 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33649.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33649.log.gz 2024-04-26T21:11:02.879 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33689.log 2024-04-26T21:11:02.879 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-26T21:11:02.880 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46877.log 2024-04-26T21:11:02.880 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38533.log.gz 2024-04-26T21:11:02.880 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38355.log 2024-04-26T21:11:02.880 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45896.log.gz 2024-04-26T21:11:02.880 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46877.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46877.log.gz 2024-04-26T21:11:02.880 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41507.log 2024-04-26T21:11:02.881 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33689.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-26T21:11:02.882 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33689.log.gz 2024-04-26T21:11:02.882 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33569.log 2024-04-26T21:11:02.883 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45361.log 2024-04-26T21:11:02.883 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38355.log.gz 2024-04-26T21:11:02.883 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46312.log 2024-04-26T21:11:02.883 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41507.log.gz 2024-04-26T21:11:02.883 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45361.log: gzip -5 --verbose 88.9% -- -- replaced with /var/log/ceph/ceph-client.admin.45361.log.gz /var/log/ceph/ceph-client.admin.44493.log 2024-04-26T21:11:02.883 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-26T21:11:02.885 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39615.log 2024-04-26T21:11:02.885 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46312.log.gz 2024-04-26T21:11:02.885 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43460.log 2024-04-26T21:11:02.885 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44493.log.gz 2024-04-26T21:11:02.886 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39615.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39615.log.gz 2024-04-26T21:11:02.886 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38532.log 2024-04-26T21:11:02.887 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.33569.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33569.log.gz 2024-04-26T21:11:02.887 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33609.log 2024-04-26T21:11:02.887 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33609.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33609.log.gz 2024-04-26T21:11:02.889 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33438.log 2024-04-26T21:11:02.889 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48944.log 2024-04-26T21:11:02.890 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43460.log.gz 2024-04-26T21:11:02.890 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46961.log 2024-04-26T21:11:02.890 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38532.log.gz 2024-04-26T21:11:02.891 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41373.log 2024-04-26T21:11:02.891 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48944.log.gz 2024-04-26T21:11:02.892 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33438.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33438.log.gz 2024-04-26T21:11:02.892 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46011.log 2024-04-26T21:11:02.892 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46961.log.gz 2024-04-26T21:11:02.893 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44144.log 2024-04-26T21:11:02.893 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41373.log.gz 2024-04-26T21:11:02.894 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46011.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T21:11:02.894 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46011.log.gz 2024-04-26T21:11:02.895 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44144.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44144.log.gz 2024-04-26T21:11:02.895 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41328.log 2024-04-26T21:11:02.896 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38852.log 2024-04-26T21:11:02.897 INFO:teuthology.orchestra.run.smithi026.stderr: 84.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T21:11:02.897 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39208.log 2024-04-26T21:11:02.898 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41328.log.gz 2024-04-26T21:11:02.899 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38852.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38852.log.gz 2024-04-26T21:11:02.899 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47956.log 2024-04-26T21:11:02.900 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39208.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39208.log.gz 2024-04-26T21:11:02.900 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27213.log 2024-04-26T21:11:02.901 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40508.log 2024-04-26T21:11:02.901 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47956.log.gz 2024-04-26T21:11:02.902 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph.tmp-client.admin.27213.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27213.log.gz 2024-04-26T21:11:02.902 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47830.log 2024-04-26T21:11:02.903 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.40508.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40508.log.gz 2024-04-26T21:11:02.904 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48776.log 2024-04-26T21:11:02.905 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47746.log 2024-04-26T21:11:02.905 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47830.log.gz 2024-04-26T21:11:02.906 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37935.log 2024-04-26T21:11:02.906 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48776.log.gz 2024-04-26T21:11:02.907 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47746.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47746.log.gz 2024-04-26T21:11:02.908 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-26T21:11:02.908 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.37935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42878.log 2024-04-26T21:11:02.909 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37935.log.gz 2024-04-26T21:11:02.910 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38525.log 2024-04-26T21:11:02.911 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44102.log 88.1% 2024-04-26T21:11:02.911 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42878.log.gz 2024-04-26T21:11:02.912 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43502.log 2024-04-26T21:11:02.912 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38525.log.gz 2024-04-26T21:11:02.913 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48650.log 2024-04-26T21:11:02.913 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44102.log.gz 2024-04-26T21:11:02.914 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48398.log 2024-04-26T21:11:02.915 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43502.log.gz 2024-04-26T21:11:02.916 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41285.log 2024-04-26T21:11:02.916 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48650.log.gz 2024-04-26T21:11:02.917 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38229.log 2024-04-26T21:11:02.917 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48398.log.gz 2024-04-26T21:11:02.918 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39993.log 2024-04-26T21:11:02.918 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41285.log.gz 2024-04-26T21:11:02.919 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39867.log 2024-04-26T21:11:02.919 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38229.log.gz 2024-04-26T21:11:02.920 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38868.log 2024-04-26T21:11:02.920 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39993.log.gz 2024-04-26T21:11:02.922 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39867.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T21:11:02.922 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39867.log.gz 2024-04-26T21:11:02.923 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38868.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38868.log.gz 2024-04-26T21:11:02.923 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37977.log 2024-04-26T21:11:02.924 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45129.log 2024-04-26T21:11:02.925 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.37977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38187.log 2024-04-26T21:11:02.926 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T21:11:02.926 INFO:teuthology.orchestra.run.smithi026.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37977.log.gz 2024-04-26T21:11:02.926 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.45129.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45129.log.gz 2024-04-26T21:11:02.927 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39573.log 2024-04-26T21:11:02.928 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33607.log 2024-04-26T21:11:02.928 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38187.log.gz 2024-04-26T21:11:02.929 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42452.log 2024-04-26T21:11:02.929 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39573.log.gz 2024-04-26T21:11:02.930 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.33607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39187.log 2024-04-26T21:11:02.930 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33607.log.gz 2024-04-26T21:11:02.931 INFO:teuthology.orchestra.run.smithi086.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-26T21:11:02.931 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44749.log 2024-04-26T21:11:02.931 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42452.log.gz 2024-04-26T21:11:02.932 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42374.log 2024-04-26T21:11:02.932 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39187.log.gz 2024-04-26T21:11:02.933 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44749.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44749.log.gz 2024-04-26T21:11:02.933 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41969.log 2024-04-26T21:11:02.935 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41099.log 2024-04-26T21:11:02.935 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42374.log.gz 2024-04-26T21:11:02.936 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40173.log 2024-04-26T21:11:02.936 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41969.log.gz 2024-04-26T21:11:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41099.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41099.log.gz 2024-04-26T21:11:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-26T21:11:02.938 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44400.log 2024-04-26T21:11:02.938 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40173.log.gz 2024-04-26T21:11:02.939 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42332.log 2024-04-26T21:11:02.940 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42536.log 2024-04-26T21:11:02.940 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44400.log.gz 2024-04-26T21:11:02.941 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38145.log 2024-04-26T21:11:02.942 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42332.log.gz 2024-04-26T21:11:02.942 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42536.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42536.log.gz 2024-04-26T21:11:02.943 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39699.log 2024-04-26T21:11:02.944 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47601.log 2024-04-26T21:11:02.945 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38145.log.gz 2024-04-26T21:11:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39699.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39699.log.gz 2024-04-26T21:11:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38528.log 2024-04-26T21:11:02.947 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38537.log 2024-04-26T21:11:02.947 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47601.log.gz 2024-04-26T21:11:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38528.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38528.log.gz 2024-04-26T21:11:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41927.log 2024-04-26T21:11:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38103.log 2024-04-26T21:11:02.951 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38537.log.gz 2024-04-26T21:11:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41927.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41927.log.gz 2024-04-26T21:11:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40035.log 2024-04-26T21:11:02.952 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48860.log 2024-04-26T21:11:02.952 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38103.log.gz 2024-04-26T21:11:02.953 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41243.log 2024-04-26T21:11:02.953 INFO:teuthology.orchestra.run.smithi026.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.40035.log.gz 2024-04-26T21:11:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48860.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-26T21:11:02.955 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48860.log.gz 2024-04-26T21:11:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41243.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41243.log.gz 2024-04-26T21:11:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43418.log 2024-04-26T21:11:02.957 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40661.log 2024-04-26T21:11:02.959 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43418.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43418.log.gz 2024-04-26T21:11:02.960 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37893.log 2024-04-26T21:11:02.962 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39825.log 2024-04-26T21:11:02.962 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.37893.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40661.log.gz 2024-04-26T21:11:02.962 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37893.log.gz 2024-04-26T21:11:02.962 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41885.log 2024-04-26T21:11:02.964 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48986.log 2024-04-26T21:11:02.964 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39825.log.gz 2024-04-26T21:11:02.965 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41885.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41885.log.gz 2024-04-26T21:11:02.965 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37690.log 2024-04-26T21:11:02.967 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48986.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-26T21:11:02.967 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48986.log.gz 2024-04-26T21:11:02.967 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.37690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39121.log 2024-04-26T21:11:02.967 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37690.log.gz 2024-04-26T21:11:02.968 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-26T21:11:02.969 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42494.log 2024-04-26T21:11:02.970 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39121.log.gz 2024-04-26T21:11:02.971 INFO:teuthology.orchestra.run.smithi086.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-26T21:11:02.971 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42752.log 2024-04-26T21:11:02.972 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41759.log 2024-04-26T21:11:02.972 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42494.log.gz 2024-04-26T21:11:02.973 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42752.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42752.log.gz 2024-04-26T21:11:02.975 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41759.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41759.log.gz 2024-04-26T21:11:02.975 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39205.log 2024-04-26T21:11:02.976 INFO:teuthology.orchestra.run.smithi086.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-26T21:11:02.977 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.40965.log 2024-04-26T21:11:02.978 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38864.log 2024-04-26T21:11:02.979 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39205.log.gz 2024-04-26T21:11:02.979 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.2.40965.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.40965.log.gz 2024-04-26T21:11:02.980 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38864.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38864.log.gz 2024-04-26T21:11:02.981 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44833.log 2024-04-26T21:11:02.983 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-26T21:11:02.985 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44833.log: gzip 88.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.44833.log.gz --verbose 2024-04-26T21:11:02.985 INFO:teuthology.orchestra.run.smithi026.stderr: -- /var/log/ceph/ceph-client.admin.44665.log 2024-04-26T21:11:02.986 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42206.log 2024-04-26T21:11:02.987 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44665.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44665.log.gz 2024-04-26T21:11:02.989 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42206.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42206.log.gz 2024-04-26T21:11:02.990 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48356.log 2024-04-26T21:11:02.990 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44060.log 2024-04-26T21:11:02.992 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48356.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48356.log.gz 2024-04-26T21:11:02.993 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44060.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44060.log.gz 2024-04-26T21:11:02.993 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47087.log 2024-04-26T21:11:02.995 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39489.log 2024-04-26T21:11:02.996 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47087.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47087.log.gz 2024-04-26T21:11:02.998 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39489.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39489.log.gz 2024-04-26T21:11:02.998 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47701.log 2024-04-26T21:11:03.000 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37787.log 2024-04-26T21:11:03.001 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47701.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47701.log.gz 2024-04-26T21:11:03.003 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.37787.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37787.log.gz 2024-04-26T21:11:03.003 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47788.log 2024-04-26T21:11:03.005 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42836.log 2024-04-26T21:11:03.006 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47788.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47788.log.gz 2024-04-26T21:11:03.007 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42836.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42836.log.gz 2024-04-26T21:11:03.008 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43376.log 2024-04-26T21:11:03.010 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39657.log 2024-04-26T21:11:03.011 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43376.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43376.log.gz 2024-04-26T21:11:03.012 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39657.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39657.log.gz 2024-04-26T21:11:03.013 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38871.log 2024-04-26T21:11:03.015 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41801.log 2024-04-26T21:11:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38871.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38871.log.gz 2024-04-26T21:11:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41801.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41801.log.gz 2024-04-26T21:11:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38523.log 2024-04-26T21:11:03.020 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45939.log 2024-04-26T21:11:03.021 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38523.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38523.log.gz 2024-04-26T21:11:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45939.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.45939.log.gz 2024-04-26T21:11:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45737.log 2024-04-26T21:11:03.025 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41633.log 2024-04-26T21:11:03.026 INFO:teuthology.orchestra.run.smithi026.stderr: 91.5%/var/log/ceph/ceph-client.admin.45737.log: -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-26T21:11:03.026 INFO:teuthology.orchestra.run.smithi026.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.45737.log.gz 2024-04-26T21:11:03.026 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43688.log 2024-04-26T21:11:03.028 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41633.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-26T21:11:03.028 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41633.log.gz 2024-04-26T21:11:03.029 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46259.log 2024-04-26T21:11:03.030 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43688.log.gz 2024-04-26T21:11:03.032 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.46259.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.46259.log.gz 2024-04-26T21:11:03.032 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39741.log 2024-04-26T21:11:03.034 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39115.log 2024-04-26T21:11:03.035 INFO:teuthology.orchestra.run.smithi086.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-26T21:11:03.035 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39741.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39741.log.gz 2024-04-26T21:11:03.037 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39115.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39115.log.gz 2024-04-26T21:11:03.037 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38865.log 2024-04-26T21:11:03.039 INFO:teuthology.orchestra.run.smithi026.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-26T21:11:03.039 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39405.log 2024-04-26T21:11:03.040 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38865.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38865.log.gz 2024-04-26T21:11:03.040 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48566.log 2024-04-26T21:11:03.042 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47517.log 2024-04-26T21:11:03.042 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39405.log.gz 2024-04-26T21:11:03.043 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44941.log 2024-04-26T21:11:03.043 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48566.log.gz 2024-04-26T21:11:03.044 INFO:teuthology.orchestra.run.smithi086.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-26T21:11:03.044 INFO:teuthology.orchestra.run.smithi086.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-26T21:11:03.044 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47517.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47517.log.gz 2024-04-26T21:11:03.044 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43891.log 2024-04-26T21:11:03.045 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42053.log 2024-04-26T21:11:03.045 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44941.log.gz 2024-04-26T21:11:03.047 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43891.log: gzip 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz -5 2024-04-26T21:11:03.047 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39169.log 2024-04-26T21:11:03.047 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43891.log.gz 2024-04-26T21:11:03.048 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47872.log 88.6% 2024-04-26T21:11:03.048 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42053.log.gz 2024-04-26T21:11:03.049 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41464.log 2024-04-26T21:11:03.049 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39169.log.gz 2024-04-26T21:11:03.050 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45417.log 2024-04-26T21:11:03.050 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47872.log.gz 2024-04-26T21:11:03.051 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37346.log 2024-04-26T21:11:03.052 INFO:teuthology.orchestra.run.smithi086.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-26T21:11:03.052 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41464.log.gz 2024-04-26T21:11:03.052 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38536.log 2024-04-26T21:11:03.053 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45417.log.gz 2024-04-26T21:11:03.054 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.37346.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37346.log.gz 2024-04-26T21:11:03.054 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38535.log 2024-04-26T21:11:03.055 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-26T21:11:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:real 0m0.227s 2024-04-26T21:11:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:user 0m0.894s 2024-04-26T21:11:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:sys 0m0.172s 2024-04-26T21:11:03.056 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46471.log 2024-04-26T21:11:03.056 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38536.log.gz 2024-04-26T21:11:03.057 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41591.log 2024-04-26T21:11:03.057 INFO:teuthology.orchestra.run.smithi026.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38535.log.gz 2024-04-26T21:11:03.058 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47475.log 2024-04-26T21:11:03.059 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46471.log.gz 2024-04-26T21:11:03.059 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41591.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41591.log.gz 2024-04-26T21:11:03.059 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45208.log 2024-04-26T21:11:03.060 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47475.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47475.log.gz 2024-04-26T21:11:03.060 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48188.log 2024-04-26T21:11:03.061 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46198.log 2024-04-26T21:11:03.062 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45208.log.gz 2024-04-26T21:11:03.066 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41717.log 2024-04-26T21:11:03.067 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48188.log.gz 2024-04-26T21:11:03.067 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47003.log 2024-04-26T21:11:03.067 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46198.log.gz 2024-04-26T21:11:03.067 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45790.log 2024-04-26T21:11:03.067 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41717.log.gz 2024-04-26T21:11:03.067 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41201.log 2024-04-26T21:11:03.067 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47003.log.gz 2024-04-26T21:11:03.067 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38873.log 2024-04-26T21:11:03.068 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45790.log.gz 2024-04-26T21:11:03.070 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43976.log 2024-04-26T21:11:03.070 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41201.log.gz 2024-04-26T21:11:03.071 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38873.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38873.log.gz 2024-04-26T21:11:03.071 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43250.log 2024-04-26T21:11:03.072 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41675.log 2024-04-26T21:11:03.073 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43976.log.gz 2024-04-26T21:11:03.073 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37831.log 2024-04-26T21:11:03.073 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43250.log.gz 2024-04-26T21:11:03.075 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41675.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48314.log 2024-04-26T21:11:03.075 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41675.log.gz 2024-04-26T21:11:03.076 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.37831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44018.log 2024-04-26T21:11:03.076 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37831.log.gz 2024-04-26T21:11:03.077 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44186.log 2024-04-26T21:11:03.077 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48314.log.gz 2024-04-26T21:11:03.078 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46139.log 2024-04-26T21:11:03.078 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44018.log.gz 2024-04-26T21:11:03.079 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47559.log 2024-04-26T21:11:03.080 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44186.log.gz 2024-04-26T21:11:03.081 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42710.log 2024-04-26T21:11:03.081 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46139.log.gz 2024-04-26T21:11:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43334.log 2024-04-26T21:11:03.082 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47559.log.gz 2024-04-26T21:11:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42710.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46514.log 2024-04-26T21:11:03.084 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42710.log.gz 2024-04-26T21:11:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39951.log 2024-04-26T21:11:03.085 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43334.log.gz 2024-04-26T21:11:03.086 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43124.log 2024-04-26T21:11:03.086 INFO:teuthology.orchestra.run.smithi026.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46514.log.gz 2024-04-26T21:11:03.087 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41420.log 2024-04-26T21:11:03.087 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39951.log.gz 2024-04-26T21:11:03.088 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46699.log 2024-04-26T21:11:03.088 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43124.log.gz 2024-04-26T21:11:03.089 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48902.log 2024-04-26T21:11:03.090 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41420.log.gz 2024-04-26T21:11:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46699.log: gzip -5 --verbose 90.4% -- -- replaced with /var/log/ceph/ceph-client.admin.46699.log.gz 2024-04-26T21:11:03.090 INFO:teuthology.orchestra.run.smithi026.stderr: /var/log/ceph/ceph-client.admin.37442.log 2024-04-26T21:11:03.091 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48692.log 2024-04-26T21:11:03.092 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48902.log.gz/var/log/ceph/ceph-client.admin.37442.log: 2024-04-26T21:11:03.092 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47914.log 2024-04-26T21:11:03.093 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37442.log.gz 2024-04-26T21:11:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48692.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48692.log.gz 2024-04-26T21:11:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39783.log 2024-04-26T21:11:03.095 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47223.log 2024-04-26T21:11:03.096 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47914.log.gz 2024-04-26T21:11:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48734.log 2024-04-26T21:11:03.097 INFO:teuthology.orchestra.run.smithi026.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.39783.log.gz 2024-04-26T21:11:03.099 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47223.log: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47265.log 2024-04-26T21:11:03.099 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47223.log.gz 2024-04-26T21:11:03.099 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42095.log 2024-04-26T21:11:03.099 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48734.log.gz 2024-04-26T21:11:03.100 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42620.log 2024-04-26T21:11:03.100 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47265.log.gz 2024-04-26T21:11:03.101 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49046.log 2024-04-26T21:11:03.101 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42095.log.gz 2024-04-26T21:11:03.102 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44791.log 2024-04-26T21:11:03.102 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42620.log.gz 2024-04-26T21:11:03.103 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46752.log 2024-04-26T21:11:03.104 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49046.log.gz 2024-04-26T21:11:03.105 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49087.log 2024-04-26T21:11:03.105 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44791.log.gz 2024-04-26T21:11:03.106 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46752.log: 90.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43586.log 2024-04-26T21:11:03.106 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46752.log.gz 2024-04-26T21:11:03.108 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38863.log 2024-04-26T21:11:03.109 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49087.log.gz 2024-04-26T21:11:03.109 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47391.log 2024-04-26T21:11:03.109 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43586.log.gz 2024-04-26T21:11:03.110 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37523.log 2024-04-26T21:11:03.110 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38863.log.gz 2024-04-26T21:11:03.111 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33504.log 2024-04-26T21:11:03.111 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47391.log.gz 2024-04-26T21:11:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.37523.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37523.log.gz 2024-04-26T21:11:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46097.log 2024-04-26T21:11:03.113 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.33504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39531.log 2024-04-26T21:11:03.115 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33504.log.gz 2024-04-26T21:11:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46097.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46097.log.gz 2024-04-26T21:11:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44875.log 2024-04-26T21:11:03.115 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39180.log 2024-04-26T21:11:03.116 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39531.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39531.log.gz 2024-04-26T21:11:03.116 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43544.log 2024-04-26T21:11:03.116 INFO:teuthology.orchestra.run.smithi026.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44875.log.gz 2024-04-26T21:11:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44228.log 2024-04-26T21:11:03.118 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39180.log.gz 2024-04-26T21:11:03.119 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42794.log 2024-04-26T21:11:03.119 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43544.log.gz 2024-04-26T21:11:03.120 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38061.log 2024-04-26T21:11:03.120 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44228.log.gz 2024-04-26T21:11:03.121 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46365.log 2024-04-26T21:11:03.122 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42794.log.gz 2024-04-26T21:11:03.122 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37737.log 2024-04-26T21:11:03.123 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38061.log.gz 2024-04-26T21:11:03.123 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48146.log 2024-04-26T21:11:03.125 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.37737.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.46365.log.gz 2024-04-26T21:11:03.125 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39447.log 2024-04-26T21:11:03.125 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37737.log.gz 2024-04-26T21:11:03.126 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48818.log 2024-04-26T21:11:03.126 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48146.log.gz 2024-04-26T21:11:03.127 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44356.log 2024-04-26T21:11:03.128 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39447.log.gz 2024-04-26T21:11:03.128 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41843.log 2024-04-26T21:11:03.128 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48818.log.gz 2024-04-26T21:11:03.130 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43166.log 2024-04-26T21:11:03.130 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44356.log.gz 2024-04-26T21:11:03.131 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48482.log 2024-04-26T21:11:03.131 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41843.log.gz 2024-04-26T21:11:03.132 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43646.log 2024-04-26T21:11:03.132 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43166.log.gz 2024-04-26T21:11:03.133 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42011.log 2024-04-26T21:11:03.134 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48482.log.gz 2024-04-26T21:11:03.134 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38313.log 2024-04-26T21:11:03.135 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43646.log.gz 2024-04-26T21:11:03.136 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42011.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42011.log.gz 2024-04-26T21:11:03.136 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-26T21:11:03.137 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42956.log 2024-04-26T21:11:03.138 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38313.log.gz 2024-04-26T21:11:03.139 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48272.log 2024-04-26T21:11:03.139 INFO:teuthology.orchestra.run.smithi026.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-26T21:11:03.139 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44581.log 2024-04-26T21:11:03.140 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42956.log.gz 2024-04-26T21:11:03.141 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47045.log 2024-04-26T21:11:03.141 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48272.log.gz 2024-04-26T21:11:03.142 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42290.log 2024-04-26T21:11:03.142 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44581.log.gz 2024-04-26T21:11:03.143 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47349.log 2024-04-26T21:11:03.143 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47045.log.gz 2024-04-26T21:11:03.144 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42290.log: gzip -5 88.9% --verbose -- replaced with /var/log/ceph/ceph-client.admin.42290.log.gz -- 2024-04-26T21:11:03.145 INFO:teuthology.orchestra.run.smithi026.stderr: /var/log/ceph/ceph-client.admin.48230.log 2024-04-26T21:11:03.145 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43082.log 2024-04-26T21:11:03.146 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47349.log.gz 2024-04-26T21:11:03.147 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47643.log 2024-04-26T21:11:03.147 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48230.log.gz 2024-04-26T21:11:03.148 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43082.log: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46567.log 2024-04-26T21:11:03.148 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43082.log.gz 2024-04-26T21:11:03.149 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48608.log 2024-04-26T21:11:03.150 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47643.log.gz 2024-04-26T21:11:03.151 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45528.log 2024-04-26T21:11:03.151 INFO:teuthology.orchestra.run.smithi026.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.46567.log.gz 2024-04-26T21:11:03.151 INFO:teuthology.orchestra.run.smithi026.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-26T21:11:03.152 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48608.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48608.log.gz 2024-04-26T21:11:03.152 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48524.log 2024-04-26T21:11:03.153 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49132.log 2024-04-26T21:11:03.153 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45528.log.gz 2024-04-26T21:11:03.154 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48524.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48524.log.gz 2024-04-26T21:11:03.154 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48440.log 2024-04-26T21:11:03.155 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39210.log 2024-04-26T21:11:03.155 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49132.log.gz 2024-04-26T21:11:03.156 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43933.log 2024-04-26T21:11:03.156 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48440.log.gz 2024-04-26T21:11:03.157 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45694.log 2024-04-26T21:11:03.158 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39210.log.gz 2024-04-26T21:11:03.158 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43933.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43933.log.gz 2024-04-26T21:11:03.158 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47181.log 2024-04-26T21:11:03.159 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44707.log 2024-04-26T21:11:03.159 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45694.log.gz 2024-04-26T21:11:03.160 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47181.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47181.log.gz 2024-04-26T21:11:03.160 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38854.log 2024-04-26T21:11:03.161 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42998.log 2024-04-26T21:11:03.161 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44707.log.gz 2024-04-26T21:11:03.162 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38271.log 2024-04-26T21:11:03.162 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38854.log.gz 2024-04-26T21:11:03.163 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37623.log 2024-04-26T21:11:03.163 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42998.log.gz 2024-04-26T21:11:03.164 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46055.log 2024-04-26T21:11:03.164 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38271.log.gz 2024-04-26T21:11:03.165 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.37623.log: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-26T21:11:03.165 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41549.log 2024-04-26T21:11:03.165 INFO:teuthology.orchestra.run.smithi026.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.37623.log.gz 2024-04-26T21:11:03.166 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44312.log 2024-04-26T21:11:03.166 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46055.log.gz 2024-04-26T21:11:03.167 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39909.log 2024-04-26T21:11:03.168 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41549.log.gz 2024-04-26T21:11:03.168 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44312.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44312.log.gz 2024-04-26T21:11:03.168 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44539.log 2024-04-26T21:11:03.169 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.39909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45571.log 2024-04-26T21:11:03.169 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39909.log.gz 2024-04-26T21:11:03.170 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43040.log 2024-04-26T21:11:03.170 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44539.log.gz 2024-04-26T21:11:03.171 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43810.log 2024-04-26T21:11:03.171 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43040.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.45571.log.gz 2024-04-26T21:11:03.172 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43040.log.gz 2024-04-26T21:11:03.172 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38019.log 2024-04-26T21:11:03.173 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33239.log 2024-04-26T21:11:03.173 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43810.log.gz 2024-04-26T21:11:03.174 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.38019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37568.log 2024-04-26T21:11:03.174 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38019.log.gz 2024-04-26T21:11:03.175 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.33239.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.33239.log.gz 2024-04-26T21:11:03.176 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.37568.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37568.log.gz 2024-04-26T21:11:03.199 INFO:teuthology.orchestra.run.smithi026.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-26T21:11:03.287 INFO:teuthology.orchestra.run.smithi026.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-26T21:11:03.291 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-04-26T21:11:03.291 INFO:teuthology.orchestra.run.smithi026.stderr:real 0m0.463s 2024-04-26T21:11:03.291 INFO:teuthology.orchestra.run.smithi026.stderr:user 0m1.167s 2024-04-26T21:11:03.291 INFO:teuthology.orchestra.run.smithi026.stderr:sys 0m0.762s 2024-04-26T21:11:03.292 INFO:tasks.ceph:Archiving logs... 2024-04-26T21:11:03.293 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675194/remote/smithi026/log 2024-04-26T21:11:03.293 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T21:11:03.698 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675194/remote/smithi086/log 2024-04-26T21:11:03.699 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T21:11:03.891 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T21:11:03.902 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T21:11:03.902 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T21:11:03.910 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T21:11:04.046 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T21:11:04.056 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T21:11:04.056 DEBUG:teuthology.orchestra.run.smithi026:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:11:04.064 DEBUG:teuthology.orchestra.run.smithi086:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:11:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:11:04.119 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:11:04.124 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:11:04.124 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-04-26T21:11:04.124 INFO:teuthology.orchestra.run.smithi026.stdout:^* hv01.front.sepia.ceph.com 3 6 377 22 +1007us[+1002us] +/- 51ms 2024-04-26T21:11:04.124 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 22 -1218us[-1223us] +/- 79ms 2024-04-26T21:11:04.125 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 -516us[ -516us] +/- 92ms 2024-04-26T21:11:04.125 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:11:04.129 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:11:04.129 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-04-26T21:11:04.129 INFO:teuthology.orchestra.run.smithi086.stdout:^* hv01.front.sepia.ceph.com 3 6 377 22 +1009us[+1014us] +/- 51ms 2024-04-26T21:11:04.129 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 22 -1193us[-1189us] +/- 80ms 2024-04-26T21:11:04.129 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 -503us[ -498us] +/- 92ms 2024-04-26T21:11:04.129 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:11:04.131 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T21:11:04.140 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T21:11:04.140 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T21:11:04.149 DEBUG:teuthology.orchestra.run.smithi026:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T21:11:04.364 DEBUG:teuthology.orchestra.run.smithi086:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T21:11:04.554 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:11:04.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:11:04.651 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:11:04.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:11:04.744 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T21:11:04.753 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T21:11:04.762 INFO:teuthology.task.internal:Duration was 665.908310 seconds 2024-04-26T21:11:04.763 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T21:11:04.772 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T21:11:04.772 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T21:11:04.780 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T21:11:04.907 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:11:04.927 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:11:05.402 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T21:11:05.402 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi026.front.sepia.ceph.com 2024-04-26T21:11:05.402 DEBUG:teuthology.orchestra.run.smithi026:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T21:11:05.472 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi086.front.sepia.ceph.com 2024-04-26T21:11:05.472 DEBUG:teuthology.orchestra.run.smithi086:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T21:11:05.549 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T21:11:05.549 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T21:11:05.557 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T21:11:05.644 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T21:11:05.645 DEBUG:teuthology.orchestra.run.smithi026:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T21:11:05.650 DEBUG:teuthology.orchestra.run.smithi086:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T21:11:05.831 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T21:11:05.843 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T21:11:05.844 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T21:11:05.930 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T21:11:06.017 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T21:11:06.027 DEBUG:teuthology.orchestra.run.smithi026:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:11:06.036 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:11:06.133 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2024-04-26T21:11:06.135 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = core 2024-04-26T21:11:06.179 DEBUG:teuthology.orchestra.run.smithi026:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:11:06.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:11:06.229 DEBUG:teuthology.orchestra.run.smithi086:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:11:06.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:11:06.283 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T21:11:06.294 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T21:11:06.294 DEBUG:teuthology.misc:Transferring archived files from smithi026:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675194/remote/smithi026 2024-04-26T21:11:06.295 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T21:11:06.396 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675194/remote/smithi086 2024-04-26T21:11:06.397 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T21:11:06.492 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T21:11:06.493 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T21:11:06.500 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T21:11:06.555 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T21:11:06.566 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T21:11:06.566 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T21:11:06.576 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T21:11:06.576 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T21:11:06.583 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T21:11:06.632 INFO:teuthology.orchestra.run.smithi026.stdout: 393797 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 21:11 /home/ubuntu/cephtest 2024-04-26T21:11:06.638 INFO:teuthology.orchestra.run.smithi086.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 21:11 /home/ubuntu/cephtest 2024-04-26T21:11:06.641 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T21:11:06.651 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T21:11:06.677 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 client.1-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/backtrace} duration: 665.908310174942 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-04-26T21:11:06.677 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T21:11:06.764 INFO:teuthology.run:pass